Your data strategy What information to collect and what you can do with it Anthony Moor Deputy Managing Editor/Interactive The Dallas Morning News [email_address]

Web 3.0: The data-driven Web Source: Chicago Tribune SEO (Google) Domain name speculation (Netscape) Search Boost The entire Web (Google AdSense) A few large sites (DoubleClick) Ad Distribution Write/Contribute (Wikipedia, Flickr) Read (Britannica online) Engagement Web services (syndication) Client/Server Architecture Dynamic (XML, Ajax, RSS) Static (HTML) State Data (mashup, widget) Page (article) Unit of Content Data powers the Web Automated mashing up of personalized content Intelligent-agent driven assembly and interactivity Everyone Geeks Audience Web 3.0 The Semantic Web Web 2.0 (2003-2010) Web 1.0 (1993-2003)

Data powers the Web

Automated mashing up of personalized content

Intelligent-agent driven assembly and interactivity

Key features of a data-driven Web Data powers Web applications Certain classes of data are becoming critical building blocks for Web 3.0 applications Structured data records published to the Web in reusable and remotely queryable formats (widgets) Leverages the Long Tail Low-cost economics and broad reach enabled by the Internet Becomes the geospatial Web (Geoweb) Merging of geographical (location-based) information with the abstract information that currently dominates the Internet Enables content remixing and repurposing (Think: mashups) Increase benefits from collective adoption not private restriction Users add value Users add their own data to that which we provide Source: Chicago Tribune

Data powers Web applications

Certain classes of data are becoming critical building blocks for Web 3.0 applications

Structured data records published to the Web in reusable and remotely queryable formats (widgets)

Leverages the Long Tail

Low-cost economics and broad reach enabled by the Internet

Becomes the geospatial Web (Geoweb)

Merging of geographical (location-based) information with the abstract information that currently dominates the Internet

Enables content remixing and repurposing (Think: mashups)

Increase benefits from collective adoption not private restriction

Users add value

Users add their own data to that which we provide

Data centers provide utility

Databases are hard to build Databases have three parts, built by different tech experts The data warehouse, where your cleaned/converted data sits The Web interface, carefully designed to be intuitive to your users The production tool, so producers can amend the data So consider them for projects that have a long shelf life Because databases persist – the data gets old So someone needs to manage and update the database

Databases have three parts, built by different tech experts

The data warehouse, where your cleaned/converted data sits

The Web interface, carefully designed to be intuitive to your users

The production tool, so producers can amend the data

So consider them for projects that have a long shelf life

Because databases persist – the data gets old

So someone needs to manage and update the database

Acquire and build databases

Interactive school guides

Create image maps about your data

Property appraisal, home sale data

Police reports and crime statistics

Voter and election guides Powered by TheVoterGuide.org

Public employee salary databases

Mashups You can do your own map mashup at Atlas Follow mashup development at Programmable Web

You can do your own map mashup at Atlas

Follow mashup development at Programmable Web

Carefully consider what to database or map Consider the time investment Ask: What job do you want to get done for your user? Crudely sketch out exactly your idea on a piece of paper then … walk through exactly the clicks a user will take to get your stuff Maps Poor: DISD bond issue map Good: Home sales map GuideLive.com Listings

Consider the time investment

Ask: What job do you want to get done for your user?

Crudely sketch out exactly your idea on a piece of paper then

… walk through exactly the clicks a user will take to get your stuff

Maps

Poor: DISD bond issue map

Good: Home sales map

GuideLive.com

Listings

Data structure powers mashups

Of course our stories don’t mashup very well. They aren’t data! Yes they are. And we ignore that fact at our peril.

Our key competitive differentiator is the data we gather every single day Our articles Images Ads Classifieds Listings Video User Content Archives Databases Blogs

Our articles

Images

Ads

Classifieds

Listings

Video

User Content

Archives

Databases

Blogs

But it’s locked, hidden and unorganized Names/Places Coppell Grapevine Mills Mall Coppell High OSU Travis Masters Emily Coker Sarah Sanders Dates/Facts Audi slid under 18-wheeler Friday morning Concepts Suicide National Merit Scholarship Motocross Untimely deaths

Names/Places

Coppell

Grapevine Mills Mall

Coppell High

OSU

Travis Masters

Emily Coker

Sarah Sanders

Dates/Facts

Audi slid under 18-wheeler

Friday morning

Concepts

Suicide

National Merit Scholarship

Motocross

Untimely deaths

So how do we get it unlocked and organized? We need some data about this data

So how do we get it unlocked and organized? We need some data about this data metadata

Metadata tells us what an article is about

Names/Places Coppell Grapevine Mills Mall Coppell High OSU Travis Masters Emily Coker Sarah Sanders Dates/Facts Audi slid under 18-wheeler Friday morning Concepts Suicide National Merit Scholarship Motocross Untimely deaths 1) So we first extract key entities and concepts

Names/Places

Coppell

Grapevine Mills Mall

Coppell High

OSU

Travis Masters

Emily Coker

Sarah Sanders

Dates/Facts

Audi slid under 18-wheeler

Friday morning

Concepts

Suicide

National Merit Scholarship

Motocross

Untimely deaths

Names/Places Coppell Grapevine Mills Mall Coppell High OSU Travis Masters Emily Coker Sarah Sanders Dates/Facts Audi slid under 18-wheeler Friday morning Concepts Suicide National Merit Scholarship Motocross Untimely deaths 2) Then filter them for relevance

Names/Places

Coppell

Grapevine Mills Mall

Coppell High

OSU

Travis Masters

Emily Coker

Sarah Sanders

Dates/Facts

Audi slid under 18-wheeler

Friday morning

Concepts

Suicide

National Merit Scholarship

Motocross

Untimely deaths

3) And finally relate them to standard categories Names/Places Coppell Grapevine Mills Mall Coppell High OSU Travis Masters Emily Coker Sarah Sanders Dates/Facts Audi slid under 18-wheeler Friday morning Concepts Suicide National Merit Scholarship Motocross Untimely deaths Names/Places Towns > Coppell Location > Grapevine Mills High Schools > Coppell OSU People > Travis Masters People > Emily Coker Sarah Sanders Dates/Facts Accidents > Auto/Truck April 25, 2008 Concepts Suicide National Merit Scholarship Motocross Teens > Deaths

Names/Places

Coppell

Grapevine Mills Mall

Coppell High

OSU

Travis Masters

Emily Coker

Sarah Sanders

Dates/Facts

Audi slid under 18-wheeler

Friday morning

Concepts

Suicide

National Merit Scholarship

Motocross

Untimely deaths

Names/Places

Towns > Coppell

Location > Grapevine Mills

High Schools > Coppell

OSU

People > Travis Masters

People > Emily Coker

Sarah Sanders

Dates/Facts

Accidents > Auto/Truck

April 25, 2008

Concepts

Suicide

National Merit Scholarship

Motocross

Teens > Deaths

The standard categories are… Names/Places Towns > Coppell Location > Grapevine Mills High Schools > Coppell OSU People > Travis Masters People > Emily Coker Sarah Sanders Dates/Facts Accidents > Auto/Truck April 25, 2008 Concepts Suicide National Merit Scholarship Motocross Teens > Deaths

Names/Places

Towns > Coppell

Location > Grapevine Mills

High Schools > Coppell

OSU

People > Travis Masters

People > Emily Coker

Sarah Sanders

Dates/Facts

Accidents > Auto/Truck

April 25, 2008

Concepts

Suicide

National Merit Scholarship

Motocross

Teens > Deaths

tax·on·o·my Pronunciation: ak-sä-nə-mē Function: noun Etymology: French taxonomie, from tax- + -nomie -nomy Date: circa 1828 1: the organizational structure of categories and attributes that define how you classify, describe and manage your data

tax·on·o·my

Pronunciation:

ak-sä-nə-mē

Function:

noun

Etymology:

French taxonomie, from tax- + -nomie -nomy

Date:

circa 1828

1: the organizational structure of categories and attributes that define how you classify, describe and manage your data

Taxonomy is the card catalog of our content A set of index terms that we manage and apply to each piece of content Terms are hierarchical : Large categories split into specific sub-categories Terms are cross-referenced, so if you look for “bucket,” you also get “pail.” A taxonomy organizes, classifies and relates our content Structuring our content just like data

A set of index terms that we manage and apply to each piece of content

Terms are hierarchical : Large categories split into specific sub-categories

Terms are cross-referenced, so if you look for “bucket,” you also get “pail.”

So why do I need it?  Faceted navigation I can click in level by level to find something (Cuisine>Asian>Chinese>Szechwan )

So why do I need it?  Faceted navigation I can click in level by level to find something (Cuisine>Asian>Chinese>Szech-wan )  Much better site search by enabling search boxes that can restrict a search to terms of a particular type or context

So why do I need it?  Faceted navigation I can click in level by level to find something (Cuisine>Asian>Chinese>Szech-wan )  Much better search by enabling search boxes that can restrict a search to terms of a particular type or context  Related information that I may not have known about (articles, photo galleries, other listings)

So why do I need it?  Faceted navigation I can click in level by level to find something (Cuisine>Asian>Chinese>Szech-wan )  Much better search by enabling search boxes that can restrict a search to terms of a particular type or context  Related information that I may not have known about (articles, photo galleries, other listings)  Multiple attributes for listings (Parking, Ambience)

So why do I need it?  Faceted navigation I can click in level by level to find something (Cuisine>Asian>Chinese>Szech-wan )  Much better search by enabling search boxes that can restrict a search to terms of a particular type or context  Related information that I may not have known about (articles, photo galleries, other listings)  Multiple attributes for listings (BYOB, Outdoor Dining)  Higher search ranking (SEO) on search engines for topic subjects, listings, classified categories Source: Chicago Tribune

‘ Hot Topic’-driven content pages provide new opportunities for keyword-targeted advertising, boost SEO rankings, increase site traffic and drive more engagement

Embedded links increase page views and boost SEO, which generates more site traffic

Geographic terms can power data mapping

… or create customized alerts where you define the geography where you want notifications to come from

Whatever you call it… it’s about describing and classifying our content Taxonomy – our standard, heirarchical categories Metadata – the keywords describing a piece of content Structured data – Information that’s been organized as above

Taxonomy – our standard, heirarchical categories

Metadata – the keywords describing a piece of content

Structured data – Information that’s been organized as above

How do I get structured data? You can do it by hand Librarians and Web producers are doing it every day But they can only do so much Should reporters be adding metadata to every story? Should line editors and/or copy editors add metadata? You can use technology IPTC AP Digital Exchange Inform Teragram NStein MetaCarta (geotagging) Serra Media (geotagging) Generate Inc. (business data)

You can do it by hand

Librarians and Web producers are doing it every day

But they can only do so much

Should reporters be adding metadata to every story?

Should line editors and/or copy editors add metadata?

You can use technology

IPTC

AP Digital Exchange

Inform

Teragram

NStein

MetaCarta (geotagging)

Serra Media (geotagging)

Generate Inc. (business data)

Elements of a data strategy

Organizing for structured data Do you need a taxonomy and data strategy? Audit your newsroom datastream Do you need a data coordinator? Gannett’s Data Desks Brought everyone who ‘does data’ together: Agate clerks, librarians, CAR staff Responsibilities: Acquiring data, creating databases, programming them for interactivity, training for building spreadsheets Do you need self-service ways the public can give you info? Yes: Web forms No: Faxes, phones, notes on paper, lists on someone’s computer

Do you need a taxonomy and data strategy?

Audit your newsroom datastream

Do you need a data coordinator?

Gannett’s Data Desks

Brought everyone who ‘does data’ together: Agate clerks, librarians, CAR staff

Responsibilities: Acquiring data, creating databases, programming them for interactivity, training for building spreadsheets

Do you need self-service ways the public can give you info?

Yes: Web forms

No: Faxes, phones, notes on paper, lists on someone’s computer

Reporting for structured data Reporting with data in mind means you gather the same fact in the same way every time Shoot every photo exactly the same way Ask the same question of every interviewee Find out all the same info from every venue Save the data Input the data alongside the story Look for databases you can bring back with your story Example: Bluegrass Instruments

Reporting with data in mind means you gather the same fact in the same way every time

Shoot every photo exactly the same way

Ask the same question of every interviewee

Find out all the same info from every venue

Save the data

Input the data alongside the story

Look for databases you can bring back with your story

Example: Bluegrass Instruments

Writing/editing for structured data Editors add and apply keywords and standard categories Bloggers (already?) tag and categorize blog posts Should you have standard categories or go with the ‘folksonomy?’

Editors add and apply keywords and standard categories

Bloggers (already?) tag and categorize blog posts

Should you have standard categories or go with the ‘folksonomy?’

Discussion http://www.slideshare.net/ajmoor