Source management for Ushahidi and SwiftRiver

Sources, collaboration and veriﬁcationA summary of relevant ﬁndings for SwiftRiver from the Ushahidi WikipediaUnderstanding Sources projectHeather Ford.SwiftRiver/Ushahidi ethnographer.

A reminder on the goals of the project• Attempting to answer the following questions: • How are Wikipedia editors choosing, verifying, editing sources? • How are they collaborating to do this? • What could we apply to Ushahidi/SwiftRiver?

Key differences between WP and Ushahidi/SwiftRiver Wikipedia Ushahidi/SwiftRiver Editors do not summarize what has Editors make summarized statements happened but provide other editorial about what has happened linked /backed functions like verifying statements and up by sources developing categories Editors choose which sources to include All sources are generally included Chosen sources must be what Wikipedians define as “reliable” which Social media sources are often the primary often does not include social media source of Ushahidi reports sources All statements must be “verified” to be Verified and unverified statements are both included (usually) reflected

Key findings relevant to SwiftRiver1. Relevance is the primary reason for choosing a source (not accuracy);2. Verification is a two-step process of reading/watching the cited source and then finding corroborating sources;3. Important to authority and transparency is an understanding of who the authors are, and what the policy/bar for entry is;4. Talk pages are critical places to alert, discuss and collaborate;5. There was incentive for collaboration around editing in breaking news stories.

1. Relevance not accuracy A number of factors are being weighed when deciding to choose a source beyond the so-called “reliability” or accuracy of a source, including:• the balance of secondary to primary sources in the article (Wikipedia policy calls for editors to use a majority of secondary sources),• whether the article is based on a breaking news story (in which case the majority of sources might have to be primary, eyewitness sources), or an article on a person (in which case their official Twitter feed, for example, may be considered a reliable source on that person);• whether the source is notable in the context of the article (misinformation can also be relevant if it is widely reported and significant to the course of events as Judith Miller’s NYT stories were for the Iraq War). Thus relevance is more appropriate a factor in deciding whether to use a source than accuracy. This becomes important when one considers how to filter results in sources searches.

2. The verification process1.Verification is a two-step process of a) reading/watching the cited source in order to check that it is being accurately reflected in the text, and b) finding sources that corroborate or oppose the statements.2.For example, the image to the right was used in the article with the caption that reflected that these were “police in civilian clothing beating a protester” until one editor watched the footage from where the screenshot originated and asserted that there was nothing in the commentary from Al Jazeera that backed up this statement about the identity of those in the image.3.Design that supports this special activity could enable editors/curators to: highlight a questionable source/statement --> further investigate by curators carefully reading/watching the original source + checking whether it is accurately reflected in the story --> checking whether the information is corroborated by other sources --> marking it as complete within a certain

3. Who are the authors? What are their policies?• Without an understanding of who edited a page and what the policies on verification or source use are, there is no way for readers to accurately gauge the veracity of the article;• On Wikipedia, we know that editors are using the principles of “verifiability, not truth”, “reliable sources” and “neutral point of view” (NPOV) to curate and edit pages and although we could know more about individual authorship, users have a general idea of veracity because of the principles in force. SwiftRiver and Ushahidi require curators to develop and reflect similar principles to readers to answer questions such as:• What are your principles for deciding which information you include/do not include? What is your POV? How can I check up on the information provided here? Are all sources going to be included? Only reliable ones? Only accurate ones? How will rumor/misinformation be represented?

4. Talk pages for collaboration• Talk pages are critical places for collaboration between editors;• It is here that they are able to reach consensus, provide space for leadership, develop speciﬁc guidelines regarding style, format, etc for the article, alert others as to breaking news, ask for help, show what they’ve done, verify sources, provide corroboration, and provide moral support (amongst others);• Could SwiftRiver/Ushahidi similarly provide a place for editors to discuss the shape of the “story” they are telling and/or discuss the the veriﬁability of a source?

5. Collaboration around breaking news alerts1.One of the most popular items offered on talk pages was for breaking news in which an editor would bring a new item to the page to achieve one or more of the following goals: a) a way to alert other editors to recently-released information relevant to the article (important especially since editors were working in different time zones and the talk page would be one of the ﬁrst places a collaborating editor would go when they woke up or starting editing); b) an ask for others to add the source to the article for them if they didn’t have the time or the access privileges, for example, if they were editing as an anonymous IP during the periods when the page was semi-protected (some editors saw the discovery of sources and posting to talk as their single task rather than editing); c) a place to “hold” sources while they were waiting for other sources or were unsure as to the relevance of the article for inclusion and were asking for opinions of other editors; d) a way to ask other editors to ﬁnd references for unsourced or incorrectly sourced information on the article.

For example Here, an editor brings a new article to the talk page, where s/he picks out the relevant facts (this was new at the time because it had been unclear up until then what the Egyptian army’s role was in the 2011 Revolution). S/he follows it up with a corroborating source and then Lihaas edits the page and marks the talk page as “done”.

Tasks Tasks generally accorded with the following workﬂow:

design implications• SwiftRiver could enable collaboration by providing a space for individuals to bring breaking news to a common page for discussion regarding relevance/ summarizing/verification/corroboration etc.• In this way, nuggets grow into larger stories that can then be connected with one another. Rather than seeing verification as a separate process happening after curatorship which is the case with the Standby Task Force, for example, verification and curating/editing happen together in small teams of people who adopt tasks, rather than predefined and continuous roles.

3 key takeaways for Ushahidi/SwiftRiver1. Use relevance rather than accuracy to ﬁlter results;2. Distinguish between statements and sources/authors;3. Decide what form/s a “report” will take.

1. Use relevance rather than accuracy to filter results• We were originally thinking of collaboration in terms of the same people doing the same work and then for the algorithm to check whether there were similarities/inconsistencies, but this requires scale and doesn’t accord with the process by which people actually collaborate;• People collaborated by checking up on cited sources, summarizing, finding relevant news, and deciding where the news fitted.

For example: Source 1 Statement Source 2 Source 3

For example: Source 1 Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Source 3

For example: Source 1 Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just released a statement saying that they were initiating a campaign to push back the “terrorists” in Aleppo.” Source 3

For example: Source 1 Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just released a statement saying that they were initiating a campaign to push back the “terrorists” in Aleppo.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.”

For example: Source 1 Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.”

For example: Source 1 (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.”

For example: (!) Aleppo_Alissa has a history Source 1 of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.”

For example: (!) Aleppo_Alissa has a history Source 1 of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.” (!)

For example: (!) Aleppo_Alissa has a history Source 1 of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.” (!) Can we really take this as a reliable source? Need more

For example: (!) Aleppo_Alissa has a history Source 1 of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to pushsiege by government back the “terrorists” in Aleppo.” forces.” (!) Source 3 Free_Syria: “Government forces just bombed the Central Children’s Hospital in Aleppo.” (!) Can we really take this as a reliable source? Need more

For example: (!) Aleppo_Alissa has a history Source 1 of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all around us in central Aleppo” Statement Source 2 Reuters: “The Syrian government just Witness Syria released a statement saying that they “Aleppo is under were initiating a campaign to push siege by government back the “terrorists” in Aleppo.” forces.” (!) Source 3(!) Sources are unreliable -- it isn’t clear who is doing the Free_Syria: “Government forces just shooting bombed the Central Children’s Hospital in Aleppo.” (!) Can we really take this as a reliable source? Need more

Distinguishing between the author and the source/refWikipedia:Ushahidi: Author (of statement) | Statement A history of contribution that is Veriﬁed/Under dispute/Unveriﬁed disputed/undisputed OR reviewed/unreviewed (i.e. new) Unreviewed/Reviewed

3. Decide what form a “report” will take• Is it just a long list? Or a map? Or a textual summary? Or a live blog-style report?• This is important because this will determine what tasks people are performing, how/whether sources should be linked into clusters, what are the most important variables/categories for ﬁltering results etc.• If it’s a map, we might want to focus veriﬁcation activities on making sure the location (of the source/author and/or event) is accurate; if it’s a live blog, then we want to enable users to provide feedback on the sources the editor has used in the story, as well as to provide the editor with ways to characterise her/his sources.

the end.

Source management for Ushahidi and SwiftRiver

by hfordsa on Aug 23, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Source management for Ushahidi and SwiftRiver Presentation Transcript