A Web Based Tool For the Detection and Analysis of Avian Influenza Outbreaks From Internet News Sources Ian Turton and Andrew Murdoch GeoVISTA Center Penn State University

Flight?

Summary Who we are? Why we did it? What is Avian Flu? What we did? How we did it? Did it work? What will we do next?

Who we are?

Why we did it?

What is Avian Flu?

What we did?

How we did it?

Did it work?

What will we do next?

Who we are? Ian Senior Research Associate in GeoVISTA Center E-Education Fellow in Dutton E-Education Institute. Andrew MGIS Student (graduated in Summer 2008) GIS Developer at ArcBridge Consulting and Training

Ian

Senior Research Associate in GeoVISTA Center

E-Education Fellow in Dutton E-Education Institute.

Andrew

MGIS Student (graduated in Summer 2008)

GIS Developer at ArcBridge Consulting and Training

What we did? Andrew needed a project for Ian’s course on web mapping, and later for his capstone project (like a dissertation). Ian had an interest in extracting geographic information from unstructured text. Picked the spread of Avian Influenza and how to map it automatically from news reports.

Andrew needed a project for Ian’s course on web mapping, and later for his capstone project (like a dissertation).

Ian had an interest in extracting geographic information from unstructured text.

Picked the spread of Avian Influenza and how to map it automatically from news reports.

What is Avian Flu? Avian flu or Bird flu is a virus Most scary strain is H5N1 but there are many others. ~60% death rate in humans. Currently no (or very limited) human to human transmission. Picture by Quiplash ! CCbyA

Avian flu or Bird flu is a virus

Most scary strain is H5N1 but there are many others.

~60% death rate in humans.

Currently no (or very limited) human to human transmission.

What we did? Designed and built a system to automatically read internet news articles and map them for us so we could gain a better understanding of how avian flu is spreading on a day to day basis. Set it running to see how it did Tweaked it a bit as we saw how it worked

Designed and built a system to automatically read internet news articles and map them for us so we could gain a better understanding of how avian flu is spreading on a day to day basis.

Set it running to see how it did

Tweaked it a bit as we saw how it worked

How we did it? Data sources Data processing tools GeoCoding tools Web Mapping tools Server Client

Data sources

Data processing tools

GeoCoding tools

Web Mapping tools

Server

Client

Data Sources Official Avian Flu sites WHO PROMED Internet News sites Google News Feedburner Collected as RSS feeds

Official Avian Flu sites

WHO

PROMED

Internet News sites

Google News

Feedburner

Collected as RSS feeds

Why does this work? Media panic/interest leads to widespread reporting of any avian flu story. Use of medical blogs like PROMED also helps overcome government restrictions on reporting. Pictures: ianstacey, quiplash, Incessantflux CCbyA

Media panic/interest leads to widespread reporting of any avian flu story.

Use of medical blogs like PROMED also helps overcome government restrictions on reporting.

What is RSS? Really Simple Syndication RDF Site Summary A standardized XML file for passing information about web log (blog) updates. You normally view RSS feeds in a feed reader We wrote programs to read for us.

Really Simple Syndication

RDF Site Summary

A standardized XML file for passing information about web log (blog) updates.

You normally view RSS feeds in a feed reader

We wrote programs to read for us.

Finding the geography Step one extract the place names, named entity extraction Custom tools Reuters’ Calais system MetaCarta GeoNames.org GeoCode the places, disambiguate London, Washington etc Custom tools MetaCarta GeoNames.org

Step one extract the place names, named entity extraction

Custom tools

Reuters’ Calais system

MetaCarta

GeoNames.org

GeoCode the places, disambiguate London, Washington etc

Custom tools

MetaCarta

GeoNames.org

Well that can’t be too hard?

Web Mapping Server Open Web Mapping Standards from the OGC (allows others to use our data). Open Source tools (we’re a poor university). Store the data points and news text in PostGIS (free spatial database). GeoServer to serve maps from the DB to web (and desktop) clients.

Open Web Mapping Standards from the OGC (allows others to use our data).

Open Source tools (we’re a poor university).

Store the data points and news text in PostGIS (free spatial database).

GeoServer to serve maps from the DB to web (and desktop) clients.

Mapping Client Remember our end users are epidemiologists not GIS users so stick with a web browser as client. OpenLayers (www.openlayers.org) JavaScript library that implements the OGC WMS and WFS standards our server uses. Allows rapid construction of an interactive web map by relative novice developers. The finished map looks a lot like a Google map so users can use it easily.

Remember our end users are epidemiologists not GIS users so stick with a web browser as client.

OpenLayers (www.openlayers.org)

JavaScript library that implements the OGC WMS and WFS standards our server uses.

Allows rapid construction of an interactive web map by relative novice developers.

The finished map looks a lot like a Google map so users can use it easily.

The Map Choice of background layers Choice of feeds http://www.experimental.geovista.psu.edu/andrew/html/avian_influenza_map.html

Zoom and Pan

Time Line We are also interested in change over time. Added SIMILE Timeline from MIT JavaScript tool allows user to scroll through time or date stamped information

We are also interested in change over time.

Added SIMILE Timeline from MIT

JavaScript tool allows user to scroll through time or date stamped information

Link to external pages

Query the map

Did it work? Yes, Well mostly, Well some of the time! We can take news feeds, geocode them and draw maps in a web browser.

Yes,

Well mostly,

Well some of the time!

We can take news feeds, geocode them and draw maps in a web browser.

What didn’t work? News sources and even medical feeds contain too many items that are about avian flu in a general sense but not actually about an outbreak. Conferences about avian flu Vaccine news Reports of other influenza outbreaks Reports of other infectious diseases (“unlike avian flu…”

News sources and even medical feeds contain too many items that are about avian flu in a general sense but not actually about an outbreak.

Conferences about avian flu

Vaccine news

Reports of other influenza outbreaks

Reports of other infectious diseases (“unlike avian flu…”

What will we do next? Improved selection of RSS items Bayesian classifier Train on a selection of “good” and “bad” items Allow user to rate articles Non-negative matrix factorization Clusters similar items based on word usage Help overcome repeated reports

Improved selection of RSS items

Bayesian classifier

Train on a selection of “good” and “bad” items

Allow user to rate articles

Non-negative matrix factorization

Clusters similar items based on word usage

Help overcome repeated reports

What will we do next? Continue to improve the GeoCoder Better disambiguation algorithms. Allow user to rate the accuracy of locations found in reports. Improve User Interface Better selection of points of interest using timeline Replace SIMILE with custom time bar

Continue to improve the GeoCoder

Better disambiguation algorithms.

Allow user to rate the accuracy of locations found in reports.

Improve User Interface

Better selection of points of interest using timeline

Replace SIMILE with custom time bar

Conclusions It is possible to construct an online automated system that can read news articles from professional and general news feeds and map them in a way that allows experts and members of the public to track the spread of avian flu outbreaks. There is still much work that can be carried out to improve this work.

It is possible to construct an online automated system that can read news articles from professional and general news feeds and map them in a way that allows experts and members of the public to track the spread of avian flu outbreaks.

There is still much work that can be carried out to improve this work.