1. Listening to the Crowd
Anahi Ayala Iacucci – Media Innovation Advisor

2. Crowdsourcing information for
elections
How to verify information collected online during elections

3. WHAT DO WE MEAN WITH
VERIFICATION AND WHY WE DO
IT?

4. Verification and falsification
The goal of verification of crowdsourced
information is to assess whether the
information reported is indeed referring to
an actual fact that happened exactly in the
same way it is reported. For this reason
the verification process needs to happen
as a multi-level approach, where all
components are analyzed separately and
then merged.

5. Types of Verification and Falsification of
crowdsourced information
Falsifying/Verifying
the context
Falsifying/Verifying
the content
Falsifying/Verifying the
source

6. What is the content?
WHO
HOW
who are the
parties involved.
Who are the
victims and who
are the
perpetrators.
WHAT
what actually
really
happened, wha
t violation has
been made
the context
and the
dynamics of the
actual fact
Content
5W1H
WHERE
WHY
why did
the fact
happened
WHEN
when the
fact
reported
happen
the
location of
the fact
reported

8. WHO
HOW
how did the fact
happen? What is
the connection in
between the
different
components of
the event?
who are the
parties involved?
Are really the
perpetrators the
once mentioned
in the report?
Who are the
victims?
Verifying the
content
WHY
The first step is to
verify the content
of the information,
looking at the
verification of the
5W1H
is there any
information on
why the fact
happen or about
the intention of
the perpetrators?
WHAT
what actually did
really happen?
Can we re-built
the entire
dynamic of the
event?
WHERE
where did the
fact happen? Is
there any
reference to the
actual place?
Addresses?
Landmarks
mentioned?
WHEN
when did the
violation happen?
Is the time of
reporting close or
not to the time of
the actual fact
reported?

9. Primary
sources
the source is
the
eyewitness of
the fact
Types of
information
sources
Tertiary sources
Secondary sources
Compilation
based upon
primary and
secondary
sources
the source is
reporting
something based
upon evidence
from primary
sources

10. Falsifying the source
The falsification of the source is often used to undermine the reliability
or reputation of a source or to gain trust from the audience
Falsifying to gain trust
This is often done to add credibility to the
information reported by attributing it to a
source that will be more trusted than the
reporter himself. For instance one common
example of this is when secondary sources
present themselves as primary sources
Falsifying to undermine
trust
On the contrary this type of
falsification is done to attribute to a
source a certain information in order
to make them look less credible, or
even personally involved in the events

11. Falsifying the context
The verification of the context is one of the most difficult things to do because
certain information may be true in the content, and the source may be a reliable
one, but the context may give to the same information a different angle and
therefore a completely different meaning. See this picture as an example:

12. THE ONLINE VERIFICATION OF
THE SOURCE

13. Social Media
ID
Direct
engagement
Media
authentication
Timing
ID of the
Content on
the Social
media
Online
verification
of the
source
Location
Influence
ID of the
Network
ID of the
trusting
network

14. The Social media ID
When getting an information from a social
media source the first thing that we need to
look at is the bio of the source (account) as
provided in its home page. In this first step
we will look only the Identity of the person to
try to re-create his/her ID based on his/her
social media bio. We can look at several
different
social
medias:
Twitter,
Facebook,
You
Tube, Flickr, Linkedin, etc.

15. Bio on Social Network
Is there a name and surname?
Is there a link to a blog or a
website?
Is there a link to the
Facebook/Twitter/LinkedIn/You
Tube/Flickr Page?
Are there pictures? Videos? Lists
of previous jobs or schools
attended?
Look for the name/surname on
Google
Check the sites and see what
content they have and what bios
they have
Check the pages and the bios there
Check all of those and see what
they can tell you about this person
Evaluate what do you find trying to understand as much as possible about this person
identity: does he/she seems to be linked to any political party? Does he/she work for a
specific company and if yes what is this company doing exactly? Does he/she have a
picture, and if yes, does the picture tells you something about his/her political/religious
affiliation? Also make sure that all information in blogs/websites/Twitter bio and so on
report the same information: any discrepancies can be a sing of an identity falsification.

16. ID of the content on social media
The second step of the online verification of the source is to
look at the content of the source as provided in its home page.
After having checked his/her Bio from the previous
section, now we will look more closely to his content on social
media, what he is talking about, what his ideas may be
regarding certain issues and so on. This process is done to add
more information to the bio of this person and to try to get a
broader picture of what this person is interested in. In this
section the methodology is the same for all social media
previously considered.

17. Twitter
Website/Blog
Linkdin page
The first step here
is to make a list of
his/her social
media, or online
presence. An
example can be:
Flickr account
Facebook
page
You tube
channel

18. In this process make sure
that you take a look at all
possible type of content
posted on social media:
videos, photos, comments to
other
people
content,
articles,
re-tweets
and
forums. Most of us have
themes running through our
interests,
but
a
tweet/messages history that
is completely consumed with
one or two issues without
the occasional personal
reference,
humor
or
reference to other topics is
suspect. Behind every social
media account should be a
personality.
Is the content
relevant to the
issue reported?
Does this
person seems to
be committed
to a certain
cause or
particularly
interested in a
certain topic?
Is this a oneissue tweeter?
The second step
is to closely look
at the content.
Does he has
strong opinions
related to
certain topics?
And if yes, what
his opinions
are?
What other
topics is he/she
posting about?
Does the content
seem reliable? Is
there content
that does not
seems to have
been supported
by other sources?

19. ID of the Social Network
Who are
his/her friends?
Do they share
his/her views
on specific
topics or not?
In this third phase we
are looking at the
social network that
this person has. In this
context we will look at
the people he/she is
interacting with and
the friends he/she has
on social networks
Do they seem
to be related or
linked to any
particular
group/party or
group?
Do they share
the same type
of content?

20. ID of the trusting network
This fourth step is related to the people this
person trust and that he seems to be influenced
by. What we are trying to see here is how to
dissect the social network and look more closely
to the network of people that this person trusts.

21. Who are the
people that this
person is retweeting most
often?
Who are the
people that
he/she is
interacting with
most often?
Twitter/Facebook
What are these
people tweeting
about and what
are the topics he
interacts with
them about?
Who is he/she
following?

22. Who are the
people that
have given
recommendatio
ns to this
person?
Who are the
people that he
has worked
for and that
he knows?
Linkdin/Blog
Who are the
people in his
network?

23. Influence
In this phase of the verification process
we will be looking at the people that
this person influence and that trust him
as a source. This information will give us
a idea of what is the influence that this
person has on his/her social media
network.

24. Who are the
people that are
re-tweeting this
person most
often?
Who are the
people that are
interacting with
this person
most often?
Twitter
What are these
people
tweeting about
and what are
the topics they
interacts with
him/her about?
Who are the
people
following
him/her?

25. Location
In this section of the verification phase we want to verify the
location of this person in the real world. The reason why we do
this is to be able to define if this person is reporting as a primary
source, a secondary source or a tertiary source. The actual
location of a person in the virtual world may be really tricky so
we need to proceed by steps.
There are two types of location services that are associated with
social media:
- Manual location settings
- Smart phone location settings

26. Manual Location settings
Some social media have the location already set up in their settings, if
the user has allowed it. For example, Twitter has this option in the
profile page:
This function though is a manual function meaning that it is the user
that manually set the place where he/she is based and where he/she is
from. Most of the time this location is the home-town of the person, and
not necessarily where he/she is tweeting from.

27. The same is valid for Facebook location settings,
that you can find in the About page of the person:

28. The Smart Phone Location settings
When people are using their smart phones to use Social
Media, they can unable the automatic location of their
messages, both on Twitter and Facebook. In this case the
location is the one of the mobile phone, so it does correspond
to the actual location of the person at the moment when the
message was sent.
Twitter
Facebook

29. Triangulation of the location
1. Check the location of
the person on the manual
settings (if available)
2. Check the location on
the smart phone settings
(if available)
3. Cross-reference
locations from section 1
and 2 (if available)
4. Cross reference the
location mentioned in the
message you are verifying
with the location
mentioned in other
messages from the same
person
Is the location the same of
the location mentioned in
the message you are
verifying?
Is the location the same of
the location mentioned in
the message you are
verifying?
Are the locations the
same?
Are those location the
same or in the same area?
NO: the person may be on
a trip or may have moved
from the original place
NO: this person is probably
a secondary or tertiary
source
NO: the person may have
been moving from the
original location
NO: if the locations are
pretty far, this person is
probably a secondary or
tertiary source
YES: pass to the
verification processes2,3,4
YES: most likely this person
is a primary source. Pass to
step 3 and 4
YES: The person is
probably a primary source
YES: This person is most
likely a primary source

30. Language/Location cross-reference
This is a second step to be done when looking for the
verification of the location of your source. In this phase you can
look at verifying whether the vocabulary, slang, accents are
correct for the location that a source might claim to be
reporting from. For instance, if the reports are coming from a
very rural area and the language is a perfect English, you may
want to further verify the source. The same if a certain source is
always reporting in a certain language and then suddenly you
start receiving reports in another language (this may indicate
that those reports are copied or taken from someone else).

31. Timing
The time is an important characteristic of the information you
are looking at. This will also tell you something about the
possible location of the person sending the message out. In
almost all social medias and blog/websites there is a time stamp
of the publication, but as in the location, this stamp can also be
misleading with regard to the actual event reported. For this
reason we need to proceed with the triangulation of this
information to get closer to an estimate of the real time of the
event and therefore to the reliability of the source.

32. Check time stamp
Is the time close to the time
of the event?
YES
NO
The source may be a
primary source
The source may still be a
primary source but just had
no access to
communication means
The source may be not a
primary source but
someone very much
following the event closely
The source may be a
secondary or tertiary
source

33. Time of account creation
This part of the verification process is intended to look at the
existence of the account before the time of the reporting. The idea is
to find out if the Twitter/Facebook handle is a new account, with only
a few tweets. If so, this makes authentication more difficult.
Arasmus notes that "the more recent, the less reliable and the more
likely it is to be an account intended to spread disinformation." In
general, the longer the Twitter/Facebook handle has been around and
the more Tweets/posts and people linked to this handle, the better.
This gives a digital trace, a history of prior evidence that can be
scrutinized for evidence of political bias, misinformation, belonging to
a certain group or another and so on.

34. Media authentication
Media authentication is the trace of this source in Media
reports: the source in fact may be quoted by trusted/bias media
outlines whether this be in the mainstream/local media or
social media space. In this case the evaluation of the reliability
of this media as a source, can tell us something about the
person we are scrutinizing. For instance a media that is bias
towards the government will use a source that is also bias
towards the government, while an independent media will most
likely rely on independent sources. Google or any other
research engine will be useful to find out if the source you are
looking for is quoted in any online medias or social network.

35. Direct engagement
This is the most difficult part of the verification process and it is
suggested to use it only after a very careful considerations of
the risks involved in a direct contact on line with the source.
For example on Twitter, you can tweet the source back and ask
them to identify the source of what they are reporting
(suggested to use DM for this). NPR's Andy Carvin has employed
this technique particularly well.
This technique will force the source of the message to either
give you more information to identify themselves or to find
additional sources to validate his/her information

36. Social
authentication
Crowdsourced
verification of
the source
Triangulation

37. Social Authentication
The Social authentication refers to the use of social networks to
identify or validate the source of the report. If you're still unsure
about the source's reliability, you can use:
- Your own social network (Twitter, Facebook, LinkedIn) -to find
out if anyone in your network know about the source's
reliability
- The entire social network to find out the reliability of the
source, for example by tweeting something like “ Does anyone
know about this person/account?”

38. Triangulation
We have seen already how triangulation of information works. The
main point is to put together all the information that we are able to
gather by looking at the previous information, and then put them
together trying to find discrepancies or compatibilities in between
them.
The triangulation of information needs to be based on 2 very
important assumptions:
- You will never have 100% certainty of the reliability of the sources
unless you know them personally
- Each information can tell you everything and the contrary of
everything.
- All the technics you are using to verify information, may also be
used by someone else to falsify your information

39. For this reason there are several things that you need to keep in mind when
applying all the methodologies described here:
1. Ask yourself all the possible questions, and never give anything for granted
2. As Arasmus notes, "remain skeptical about the reports that you receive.
Look for multiple reports from different unconnected sources." The more
independent witnesses you can get information from, the better.
3. If after doing all the necessary verifications of the report, you find out that
the user reporting an event is not necessarily the original source, can the
original source be identified and authenticated? In particular, if the original
source is found, you still need to go through the same steps highlighted
here.
4. Run multiple verifications for the same user and, even better, ask others to
run the same verification processes on the same account. Personal beliefs
and state of mind can change how we interpret the same information.
5. In general, keep in mind that the less you find about someone the less
reliable you can consider that source.

40. THE VERIFICATION OF THE
CONTENT

41. Online verification of the content
Once the source of an information has been authenticated, we need to
verify if the content is actually true. This process is very time
consuming because the description of an actual fact can be presented
in four different ways:
- It can be accurate in all aspects
- It can be not accurate but still referring to an actual fact
- It can be accurate in all factual aspects but misleading in terms of
the context
- It can be completely false
To verify the content of an information, we need to look at the so
called 5W and 1H.

42. Falsification of the Content

43. Language
Content ID on
Social
Network
Time and
place
Online
verification
of the
content
Triangulation
Pictures/Video
Follow up with
source

44. WHO
HOW
who are the
parties involved.
Who are the
victims and who
are the
perpetrators.
WHAT
the context
and the
dynamics of
the actual fact
what actually
really
happened
Content
5W1H
WHERE
WHY
the location
of the fact
reported
why did the
fact happened
WHEN
when the fact
reported
happen

45. The verification of the
content starts with the
verification of the different
components
of
the
information. For each of
the 5W1H you will need to
look at that specific
information
and
ask
yourself several questions
to make sure that each of
those correspond to the
information reported. Some
of those questions will not
be answered directly in the
report, but will need from
your part an online search
for other sources reporting
the same event. See the
Triangulation slides for
more on this.
WHO
HOW
How did the fact
happen? Is it
possible to make
connections
between the
different
components of
the event?
WHY
is there any
information on
why the fact
happen or about
the intention of
the perpetrators?
who are the
parties involved?
Who are the
perpetrators?
Who are victims?
The first step is to
verify the content
of the
information,
looking at the
5W1H
WHAT
what did actually
happen? Can we
re-built the entire
dynamic of the
event with all the
details of the
event?
WHERE
where did the
fact happen? Is
there any
reference to the
actual place?
Addresses?
Landmarks
mentioned?
WHEN
when the
violation
happen? Is the
time of reporting
close to the time
of the actual fact
reported?

46. Language
Language is a very important factor of the
information reported that you may want to look at.
Some of the particular things related to the use of
language that may give you some indication about
the veracity of the information reported are:
•Use of dialectal words
•Use of official language like “breaking news”,
“urgent”, “confirmed” etc.
•The specific vocabulary, slang, accents, which
needs to be cross checked, to see if they are correct
for the location that a source might claim to be
reporting from.

47. Is the language used
in the report neutral
or the person is using
lots of adjective and
qualitative judgments
on the information
reported?
Is the language used
in the report
technical. For
example using a lot of
journalistic language
or human right
language?
Is the language used a
proper language or a
dialect? Are there
particular
inflections, slang
words or grammatical
mistakes in the
report?

48. WHO
Is there any
indication of
who are the
people in the
video?
HOW
What is the
context and the
dynamics of the
actual fact?
WHY
Is there anything
that can tell you
what is the
motivation of
the people
involved in the
action?
If the report you
receive is also
paired with a
photographic or
video “evidence”,
you need to look
into the
photo/video
provided to search
for any clues about
the 5W1H:
WHAT
What are they
doing in the
video/picture?
Can you assess
that with no
doubts?
WHERE
is there any sign
of where the
video/picture
was taken?
WHEN
Is there any
indication of
when the
video/picture
was taken?

49. Video
What are the sounds on the
background?
Do people mention names of
people or places in the
dialogue?
Is there any indications of the
place based on the action taking
place?
• Is there any sound that you can connect with the place? For example sounds
of cars passing by, people talking in the background?
• If they do, in which context? Do those places/people means something in
the context of the action taking place?
• Can you identify buildings, signs, cars, etc., in the background?
Are there objects in the video
that may give you additional
information?
• For example, check weaponry against those known for the given country.
What is the weather like in the
video?
• Examine weather reports to confirm that the conditions shown fit with the
claimed date and time. Also looks for shadows to determine the possible
time of day that a picture was taken.
Which language is the dialogue
taking place in?
• Language used, tone of voice of the people in videos footages can also be a
good indication of the place and context.

50. Pictures
Is there an
indication of the
place where the
photo was taken? A
sign, a particular
place marker?
How are the people
in the photo
dressed? Does this
fits with the place
where the photo is
supposed to be
take?
Where is the light
coming from in the
picture? Does it
seems artificial?
If yes, look for
other pictures
on line to
make sure
that that
place really
looks like
that. Google
street view
for example is
a good way to
cross-check
streets and
public places
Search on line
for the
meaning of
those dresses:
does
particular
outfit tells
you who this
person is or
who he/she
belongs to?
Cross-check
that the
direction of
the light is
indeed in line
with the
position of
the sun at
that time (like
from above if
midday, etc)
Are there objects in
the picture that do
not fit in the overall
scene?
Is the picture very
clear and
professional or does
it looks like it was
taken with a
phone?
Look very
carefully at
the objects on
the
background,
the corners of
the image and
the floor.
There are
indeed
phones that
take very
clear
pictures, but
in general
double check
if a picture it’s
too perfect
and too clear.

51. Follow up with source
As explained before, this is the most difficult part of the
verification process and it is suggested to do it only after a very
careful considerations of the risks involved in a direct contact with
the source.
This type of verification can happen in two ways:
-Online
-Offline
The best situation is when the verificators already have contacts in
the geographic area of interest, so that they can ask them to follow
up directly/in-person to confirm the accuracy of the report.
Obviously this is not always possible, particularly in conflict zones.
In using this technic it is very important to keep in mind that the
people verifying the information on the ground may not be able to
do so directly, but still they could gather first hand witnesses
reports based on their local network.

52. On Line Contact
The verification of the information provided online is
basically based on the creation of a dialogue with the
sender of the information. If for example someone
has reported something on Twitter or Facebook you
can simply write back to them to ask for more details
about the event reported. In this case it is suggested
to do it privately (via Facebook mailing box or Twitter
DM). Before engaging in any online conversation
with anyone it is suggested also to inquiry as much
as possible if this is safe and if the person reporting
will not be put in danger by you contacting them.

53. Offline Contact
The idea of contacting personally someone is related
to the possibility to have someone on the ground that
is trusted and safe to get in contact with. If such a
contact exists then this person (or network of
persons) can be used to directly get in contact with
the reporters to inquiry about the issue reported.
Again, in this case very careful security and privacy
considerations needs to be done. The use of this
strategy to verify information is suggested as the last
resource especially because both the trusted person
on the ground and the reporter’s lives can be put in
extreme danger.

54. Time and place
The time and place of the event normally
present the most problematic issues to verify
since they are the once that can be affected
by the re-reporting of the information by
different sources. The idea here is that most
of the time we are in front of a “Chinese
whispers” type of situation and we need to
make sure that those two variables are
reported correctly.

55. Spelling: if the original language of the
reporter it is different than the one of
the report, you can find that a place may
refer to several locations due to spelling
issues. One example is Arabic, where the
numbers of letter As in the alphabet are
several, while in English there is only
one.
Time and
Place
Format of dates: the format of
dates are different form place to
place, so people may translate
09/08/12 as 9 August 2012, or as
08 of September 2012. If people
repost a message and decide to
change the format of the date, they
may not know which one of the
two version they are using.
Edited Re-Tweets: when you find
tweets reporting something try to
do as many searched as possible to
find out what the original tweet
was saying. People tend to edit retweets when they think there is
something wrong (like spelling of
names) and in this way they may
actually change a correct name.

56. Triangulation of content
The triangulation of information is basically the act of putting together all the information
you can find about the issue reported and cross-check them against the fact as reported
to you.
To do this it is suggested to start filtering all possible information sources available: social
media, websites, blogs, local and international media, TVs, and so on.
In this tasks, make sure you always remain skeptical about the reports that you receive.
Look for multiple reports from different unconnected sources, since the more independent
witnesses you can get information from, the better and the less critical the need for
identity authentication.
If after doing all the necessary verification of the report, you find out that the fact is
reported by other people but with different characteristics, like time and location for
example, are not matching, do not discard immediately the fact as false but try to identify
which version of the report is the most accurate. If you cannot find confirmation for any of
the versions, do not chose the most likely one, but make sure you report all versions and
highlight which difference they have. The verification of the context often can help in the
identification of the those partial falsifications.
Run multiple verifications for the same user and, even better, ask others to run the same
verifications processes on the same account. Personal beliefs and state of mind can
change how we interpret the same information.
In general, keep in mind that the less you find about something that less reliable you can
consider that information, and the more you have to rely on people on the ground to
confirm it.

57. Content ID on Social Network
When you are crowdsourcing information about a
specific event or place, try to look in the social
network about that same event. If you do not find
anything related to that event, search for events in
the same area or for issues that may be related to
that event.
In the social networks normally events tend to be
repeated and reported in several platforms (Twitter
and Facebook for example) so this process will also
help you identifying possible editing of the event
reported done by different users.

58. Social
authentication
The creation of
Virtual Trusted
Networks
(active and
passive)
Crowdsourced
verification of
the content
Trusted
Network on
the ground

59. Social authentication
When you are crowdsourcing information about a specific event
or place, you should start finding an online social networks
related to that specific issue. For example in Twitter, that specific
social network will be the one twitting very on a specific hashtag
(#).
You can use that social network to ask information about how
unusual the event reported is, to get a sense of how likely it is to
happened in the first place. This online community, for example,
can help you translate, triangulate, and track down key
information. They can enable you to crowdsourced verification
but you must always check and challenge what you are told.
Ask people what they think, what do they know about that area,
or about that event. You have also to remember that you must
NEVER give an indication of what your source is, if it is not
public already, and try not to give too many details of the
information reported if that information is not public already.

60. Trusted Network on the ground
When verifying information online the use of an
offline network can be very helpful. As
mentioned before, this network can be your
ears and eyes on the ground, but they can also
incur in considerable risks by doing so.
For this reason those networks need to be very
trusted, possibly people that you personally
know and it should be open only to other
people known and trusted personally.
As already said in the module, security issues
should be prioritize over everything else.

61. The creation of Virtual Trusted
Networks (active and passive)
In several occasion, the creation of a trusted virtual
network has been done by several activists around
the world to verify crowdsourced information. This
network has been created by starting with a group of
very trusted people on the ground (personally known
and contacted) and then expanded only to people
vetted by this same group of people, and only if
personally met.
To do this there are several tools that have been
used, like Skype (better if encrypted since Skype is
indeed vulnerable to monitoring by third
parties), Google groups, encrypted emails mailing
lists, etc.

62. The creation of Virtual Trusted
Networks (active and passive)
It is also possible (and suggested) when doing
crowdsourcing online to create a list of the sources
that provide information online (directly to you or via
social networks) that you can keep updated during
your project. This list can also be compiled by adding
comments to each of the accounts/people listed and
in this way starting to create your reliable list of
sources and “scoring” them according to your
verification. This list will also be very useful for all the
processes done before, and especially once you
have done the source verification for those account
and can attest the reliability of the source.

63. THE VERIFICATION OF THE
CONTEXT

64. Context Matters
The understanding of the context of an information
it is crucial to give a sense to what has really
happened.
Specifically in the context of crowdsourced
information the context become extremely
important because we may not know the source
and we may have to find out if the information is
actually true.
On the other side though, even if the fact reported
really did happened and the source is reliable, we
may be in the situation where the context of the
information can give us a completely different idea
of what is the overall picture surrounding the event.

65. Why Context matters?
People develop their opinions based on the information available to
them, and this is how we evaluate the verification processes of an
information. Sometime though if the fact is not presented it in the
right context, the same information may be evaluated in very
different ways.
The idea here is that once we have attested that a fact has
happened, we need to make sure that every fact relevant to the
story, which is a necessary part of the story, is also reported.
In the context of this module we will look at two different ways that
you can use contextualization technics to verify information:
1. Find data that can further support or give you an idea of the
veracity of the information
2. Find data that can give a meaning to the information reported

66. Factual
context
Time
and
Location
On line
Verification
the context
Global
Context

67. Time and Location
We have already seen before how time and
location can help us identify the veracity of an
information, but in this case we will look at
them
as
important
factors
for
the
contextualization of the event reported.
Sometimes the time and location of an event
make a huge different in considering the
reliability of the information reported. What we
can look at here is if the event reported makes
sense considering the time and location
reported.

68. Time
Does the time of the event make
sense for the event reported?
For example: if an attack to a street
vendor is reported, does the time
of the event reported make sense
considering the type of economic
activity?
Also, does the time gives a specific
meaning to the event? Like an
attack to a mosque on Friday? Or to
a Synagogue on Saturday?

69. Location
Does the location of the even make
sense for the type of event
reported?
For example, does the event take
place in a location where it is very
unlikely to find the actors involved?
Does the location have a specific
meaning for the type of event
reported?
Does the location have access or
closure times that do not fit with
the time of the event reported?

70. Factual context
The first thing that we need to look at when looking at the context is to try to place the
event we are analyzing in a greater picture. The picture in this page is an example. If
you think about how you would describe the picture, the context of the event is giving
you a completely different information depending on if you look at the right or left part of
the picture. The event happening is always the same though. This is what we mean
when we say that you have to look at the factual context.

71. Global context
The global context refer to the international context
around that event. Sometimes political crisis outside a
country can trigger events in a third country. One example
can be the sudden explosion of tensions in between two
neighboring countries that can push a third one to close
its borders or move contingents of the army towards the
borders.
The global context is also something related to the
important facts that are happening in the same country or
in the same location of the event reported. The national
context of a reported event in fact and when it does
happen, can have a relationship with the reasons behind
the event, that can help us understand if the event is likely
to be true or if it can be triggered by other events
happening in the same place or at the same time.

72. Is there any other event
similar to this one that has
happened in the same time
or area?
For example something very
similar reported in the same
area
Or something that could
have triggered that event
(like a retaliation or an
attack)
Like if an attack is
reported, are there other
attack been reported in the
same area before or after?
For example, massive arrests
of activists may be following
the dead of a police man
during a demonstration
Or something that could
have influenced the way
people described what they
have seen (for example
something very shocking or
inflammatory)
In this case people may report
event to you exaggerating the
event because they are in a very
emotional state of mind
following a very shocking event

73. Look at what is going on the
region, but also globally in the
world
Is there any political issues
happening that can be related to
the country? Like important
bilateral meetings? Or peace
treaties? Or important speeches
being delivered?
Are there economical issues/
factors that can have an influence in
the country? Sudden rise in
unemployment? Sudden rise in
price of oil?
There has been any events
happening in neighboring countries
that may have lead to reaction in
the place you are considering? For
example sudden rise in tension in
between two countries? Or an
economical agreement reached?

74. WHAT TO KEEP IN MIND

75. The 9 golden rules
1. Unless you see it and you
hear it, it is not 100%
verified.
2. After rule I, chose your
verification criteria in
advance and make them
public (if possible)
3. Chose your trusted network
and make clear rules for
being part of it
4. Doubt and question
everything, always
5. Be always ready to change
your mind about the
reliability of a source
6. Be always ready to change
your mind about the veracity
of an information
7. You don’t need to verify
everything. Choose you
battles
8. Always admit when you fail
or when you do a mistake
9. What you know is also what
your enemies know. Keep
this always in mind when
verifying information

76. Keep your reputation intact
Since we said that rule number one for the verification of
crowdsourced information is that unless you see and hear it, it is not
100% verified, you need to protect your reputation and your
credibility every time you use, republish, collect or advertise
crowdsourced information.
To this, you need to be very transparent about your verification
criteria and about what degree of verification you can achieve with
the methodologies you are using. This also means that you do not
have to decide just in between verified and not verified (like 0-100)
but you can also choose to use several degrees of verification in the
middle.
The main important things is that you make very clear to whoever is
taking that information from you what criteria you used and how are
you verifying your sources and information.
A second very important point, as expressed before, is that you
need to always be ready to admit when you did a mistake in your
verification process and be ready to change your protocols
according to the situation.

77. HOW TO CREATE
VERIFICATION PROTOCOLS

78. Step I: Chose what you need to verify
You do not necessarily need to verify
everything. Choose what are most important
information by looking at the typology and
amount of information that you received.
Make a considerate evaluation of your
capacity. It is better to have few very well
verified information, then a lot of quasi
verified information.

79. Step II: Choose your verification criteria
Create your criteria for verification in advance
and by looking precisely at what means you
have to verify information. Ask yourself:
1. When is that you will consider an information
verified?
2. When is that you will not consider it verified?
3. What degrees of verification are you willing
to have in the middle?

80. Step III: draft your trusted sources list
Choose very carefully who you will consider
a trusted source. This does not necessarily
need to be mainstream media or known
people.
In addition to this you need to update your
trusted sources list periodically (I would say
on a daily bases) to make sure that you add
new sources and review the trustfulness of
old sources.

81. Step IV: Draft your verification protocols and
stick to it
Once you have drafted your verification
protocols, make sure you always apply them
to all information you process. Do not allow
yourself to do exceptions to it, and make
sure that all your information are verified
following the same procedures. This is very
important to assure consistency and to be
able to always get back to your verification
process to refine where it is not working.

82. Step V: make it public
One of the main important things about your verification
protocol is to make it as public as possible. The reason
behind this is not to inform people how they can go
around it, but to assure that people do know what sort of
credibility your information have. You do not have to
make the all details public, but people using your
information need to know exactly how much they can
trust your information or not. The more they are aware of
the verification protocols you use, the more you will be
able, if necessary to explain eventual mistakes or failure
in verifying information, as well as managing
expectations about the reliability of the information
posted.

83. EXAMPLES

84. Blogging the Egyptian Election “BEE” 2010

85. U SHAHID PLATFORM

86. Step 1: what to verify
One of the main problem of verification of crowdsourced
information is related to the fact that the overload of information
can make it impossible to actually do a meaningful verification of
the information received. For this reason, the U-Shahid team’s
first step was to define concrete criteria for what types of
reports required verification. They didn’t attempt to verify all
the content that was available, but only the one that was
important or critical.
1. Reports related to an immediate threat or act of violence
2. “Grave electoral fraud” the gravity of which was assessed by
the importance of the parties involved (celebrities, known PMs,
government officials) or the gravity of the fraud (involving
violence, or massive votes exchange) or it’s location (inside the
pooling stations)

87. The report was
supported by a
video or pictures
that clearly
confirmed what was
been reported
(especially location);
At least one of the
sources of the
information was
clear and known
(i.e., 2 SMSs from
unknown sources
could not verify
each other)
The U-Shahid team also
created some criteria that
all reports need to be
checked against to be
considered as verified. If
one or more of the criteria
was met, then the report
would have been
considered verified
Messages coming
from social media
(Twitter, Facebook
and blogs) needed to
be confirmed by an
SMS, a media report
or a direct witness;
It was been reported
by two or more
independent sources
(the list of
independent sources
was created in
advance and
updated by the
team) ;

88. Step 3: Direct contact with the
source
By asking to provide a video or
picture of the event but only if
it was safe to do so.
If the report was sent
by SMS, that number
was called to verify
the person’s identity.
The witness was
asked if they
observed the event
themselves or if they
simply learned about
the event from
someone else. More
specifically, details on
the 5w1h are asked.
If the event being
reported was still
unfolding, the
witness was asked if
anyone else nearby
was able confirm the
information.
If the report came from
Twitter itself, the account of
the tweeter was reviewed.
Simple content analysis of
previous tweets and the
account holder’s bio was
carried out online, in addition
to revised the Tweeter’s
followers.
For reports that had to
be verified but did not
meet the criteria
outlines before, the
verification team was
engaging directly with
the source in 4
different ways
By tweeting back to
the original Twitter
user asking for more
information—again
using the “who did
what, to whom, how
and where” format,
asking followers to
confirm if the
information was
indeed correct.

89. Step 4: In person verification
The U-Shahid team determines whether one of their
election monitors (trained by them) was close to the area
referenced in a report that required verification. If a
monitor was on site or close enough, that person was
asked to verify the report.
The second core strategy involves inperson verification via a trusted
network of monitors on the ground:
If U-Shahid did not had any monitors in the
area, the team was activating their NGO partners
who may had monitors in the area. If so, those
individuals were asked to confirm the validity of
the report being investigated.
If no one of the two possibilities was possible, UShahid was using a network of trusted journalists
in the area to ask them to go to the place to
confirm the information for them.

90. Step 5: online triangulation
In addition to this
the U-Shahid team
was then
triangulating the
report being
investigated with
other reports they
had already
received.
If no one of the
previous possibilities
was available, the team
was using the
mainstream media and
online tools for
confirmation. They
used web-based
research to look for
any evidence that is
specific to the event
that was reported as
well as that location.
They looked for article,
blogs, social media,
video or pictures,
mainstream media
articles and TV shows
that could confirm the
information reported.

91. OUTCOMES
Ushahid received on election day November
28, 2010 during voting period 1252 reports,
documented with 287 videos and 111
images. By the end of the project on Dec 28,
2010 a total of 2700 were uploaded on the
platform with an average of reports per day
of 27 and a percentage of verified reports of
91.26%.

92. Electua 2012

101. THANK YOU
Anahi Ayala Iacucci
Media Innovation Advisor
aayala@internews.org
@anahi_ayala
@info_innovation