"Analytic Journalism:
Digital Evolution in the
Datasphere"
Tom Johnson
Managing Director
Inst. for Analytic Journalism
Santa Fe, New Mexico USA
tom@jtjohnson.com
Presentation to the
South African National Editors Forum
22 November 2010
Johannesburg, South Africa
Today’s conversation
• Journalism past [briefly] and present
• What is Analytic Journalism
• Where we think journalism is (should be?)
heading
3
What’s Changed in Journalism?
• Old:
“News” = spoken word, text, images
• Today: “News” = Data = 1s & 0s
• Global ubiquity and fluidity of data
• 5 billion cell phones [July ’10]
• Africa: 1/3 pop. have cell phones
• Metadata
• Editors  Database administrators
• Is the Mass Media dead?
4
What’s Changed in Journalism?
• Datasphere Basic Process:
Data In  Analysis  Info Out
• Former “basic process”
– Superficial research
– Gather anecdotes & quotes & AV
– Write, i.e. String together in “acceptable
form”
– Publish/broadcast soonest
– Do it again tomorrow
5
What’s Changed in Journalism
Industrial Age Journalism
– High capital investment: presses, paper, delivery,
broadcasting equipment
– Low investment in “wet ware,” i.e. employees and
their skills
Digital Age Journalism
• Low capital threshold for data, analytic tools,
“publishing”
• High educational threshold for data, analytic
tools, “publishing”
• Power shift from authority and institutional
power to sub-groups and individuals
What’s Changed in Journalism?
• Constant deadlines, but …
• Breaking news has already broken
• If people already have the “news,” what do
we have to contribute?
7
What is “Analytic Journalism”
Traditional Journalism
•Quick & Dirty research
•Gather quotes
•Harvest anecdotes
•Get some video
•String together in a 30,
60, 90-sec story
•Repeat tomorrow
8
Analytic Journalism
•Not events, but context & trends
•Define phenomena of interest as a
SYSTEM
•Employ RRAW-P process
Research  Reporting Analysis 
Write  Publish”‟
•Non-traditional analytic tools:
GIS, data mining, forensic
accounting, Social Network Analysis
Cornerstones of IAJ
• General Systems Theory
• Statistics
• Data visualization
- www.niceone.org/lab/refugees/
• Simulations
- Simtable Fire Model
- St. Marks Square
- Venice boats
- Zozobra park
Cornerstones: IAJ version of Analytic
Journalism
• General Systems Theory
1. System boundaries
2. Variables
3. Relationships between variables
4. System has goals
5. System capable of “learning”
6. System exists in an environment
• “System” can be self defined; can be
defined by outside observer(s)
• System is “scaleable”
10
Newspaper as a system
Theory of Journalistic Process
12
Data In Analysis  Info
Out
Theory. It’s important, valuable.
13
Data In  Analysis  Info Out
•Notes
•Text
•Numeric
•Images
•Charts/Graphs
•Maps
•Audio
•Video
•Atoms  Bits
•How? Who?
Theory. It’s important, valuable.
14
Data In  Analysis  Info
Out• Notes
• Text
• Numeric
• Images
• Charts/Graphs
• Maps
• Audio
• Video
• AtomsBits
How? Who?
• What are we looking
for? How can we be
surprised?
• Source
• Definition
• Context
• Estimating
• Counting
• Statistical
• Geostatistical
• Social Network Analysis
• Forensic accounting
Theory. It’s important, valuable.
15
Data In  Analysis  Info
Out• Notes
• Text
• Numeric
• Images
• Charts/Graphs
• Maps
• Audio
• Video
• AtomsBits
How?
• What are we
looking for? How
can we be
surprised?
• Source
• Definition
• Context
• Estimating
• Counting
• Statistical
• Geostatistical
• Social Network
Analysis
• Forensic
accounting
• Broadcast
• Web
• Audio
• Video
• Text
• Data visualization
• Maps
• Dynamic databases
• Archives
Theory. It’s important, valuable.
16
Data In  Analysis  Info
Out
• Notes
• Text
• Numeric
• Images
• Charts/Graphs
• Maps
• Audio
• Video
• AtomsBits
How?
• What are we
looking for? How
can we be
surprised?
• Source
• Definition
• Context
• Estimating
• Counting
• Statistical
• Geostatistical
• Social Network
Analysis
• Forensic
accounting
• Broadcast
• Web
• Audio
• Video
• Text
• Data visualization
• Maps
• Dynamic databases
• Archives
Contributed content
Contributed content
& analysis
Developing Tools for Collaboration
• Who is using wikis? Google Docs? Zoho?
• Intra-nets
• Inter- and intra-nets
Goals for Journalism Education
• Raise basic admission standards
– Applicants already can write; can do basic arithmetic
• Develop/improve tools for newsroom and citizen
journalism collaboration
– Intranets
– Wikis
– Google Docs
– Blogs, social media, Flickr, etc.
– Research customization for journalism
• Develop tools for story telling
Community & Collaboration
• Citizen Journalism and Democracy
in Africa:
“Periodismo Ciudadano y Democracia en África”: Un libro en línea de
Fackson Banda
• UK: The Guardian uses the citizen
journalist to great effect in
Expenses Scandal
• Investigate your MP’s expenses
• Oldweather.org Oldweather.org
• NYTimes Knowledge Network
El New
York Times ofrece un curso online de periodismo ciudadano
19
Online learning; online tools
• ICFJ Anywhere
Global source for high-quality, online journalism courses
taught by seasoned professionals.
• Google Refine
Google Refine is a power tool for working with messy
data, cleaning it up, transforming it from one format into
another, extending it with web services, and linking it to
databases like Freebase.
• Many Eyes
Site is set up to allow the internet community to upload
data, visualize it, and talk about their discoveries with
other people.
20
Storytelling R&D? Innovation?
21
Storytelling R&D? Innovation?
22
• Potential for “drill-down” dynamics
• Reach URLs
• Show data for statistical analysis
• Ask questions; capture answers
Tomorrow?
• Our job is to “monitor the centres of
power.”
-- Amira Haassaid
23
Tomorrow?
• In a democracy, news media
harvest data; become central
warehouse of citizen’s data
• A “public library” on steroids
• Digital resources for in-house
journalists
• Create “dashboards” to monitor gov’t.
• Potential to create specialized, non-
journalism “products”
• Provide data to app creators
Tomorrow?
• Paying for news rapidly declining
market.
• So what can journalists offer?
25
"Analytic Journalism:
Digital Evolution in the
Datasphere"
Tom Johnson
Managing Director
Inst. for Analytic Journalism
Santa Fe, New Mexico USA
tom@jtjohnson.com
Theory. It’s important, it’s valuable.
27
Nothing is as valuable as a
good theory.

Analytic Journalism: Digital Evolution in the Datasphere

  • 1.
    "Analytic Journalism: Digital Evolutionin the Datasphere" Tom Johnson Managing Director Inst. for Analytic Journalism Santa Fe, New Mexico USA tom@jtjohnson.com
  • 2.
    Presentation to the SouthAfrican National Editors Forum 22 November 2010 Johannesburg, South Africa
  • 3.
    Today’s conversation • Journalismpast [briefly] and present • What is Analytic Journalism • Where we think journalism is (should be?) heading 3
  • 4.
    What’s Changed inJournalism? • Old: “News” = spoken word, text, images • Today: “News” = Data = 1s & 0s • Global ubiquity and fluidity of data • 5 billion cell phones [July ’10] • Africa: 1/3 pop. have cell phones • Metadata • Editors  Database administrators • Is the Mass Media dead? 4
  • 5.
    What’s Changed inJournalism? • Datasphere Basic Process: Data In  Analysis  Info Out • Former “basic process” – Superficial research – Gather anecdotes & quotes & AV – Write, i.e. String together in “acceptable form” – Publish/broadcast soonest – Do it again tomorrow 5
  • 6.
    What’s Changed inJournalism Industrial Age Journalism – High capital investment: presses, paper, delivery, broadcasting equipment – Low investment in “wet ware,” i.e. employees and their skills Digital Age Journalism • Low capital threshold for data, analytic tools, “publishing” • High educational threshold for data, analytic tools, “publishing” • Power shift from authority and institutional power to sub-groups and individuals
  • 7.
    What’s Changed inJournalism? • Constant deadlines, but … • Breaking news has already broken • If people already have the “news,” what do we have to contribute? 7
  • 8.
    What is “AnalyticJournalism” Traditional Journalism •Quick & Dirty research •Gather quotes •Harvest anecdotes •Get some video •String together in a 30, 60, 90-sec story •Repeat tomorrow 8 Analytic Journalism •Not events, but context & trends •Define phenomena of interest as a SYSTEM •Employ RRAW-P process Research  Reporting Analysis  Write  Publish”‟ •Non-traditional analytic tools: GIS, data mining, forensic accounting, Social Network Analysis
  • 9.
    Cornerstones of IAJ •General Systems Theory • Statistics • Data visualization - www.niceone.org/lab/refugees/ • Simulations - Simtable Fire Model - St. Marks Square - Venice boats - Zozobra park
  • 10.
    Cornerstones: IAJ versionof Analytic Journalism • General Systems Theory 1. System boundaries 2. Variables 3. Relationships between variables 4. System has goals 5. System capable of “learning” 6. System exists in an environment • “System” can be self defined; can be defined by outside observer(s) • System is “scaleable” 10
  • 11.
  • 12.
    Theory of JournalisticProcess 12 Data In Analysis  Info Out
  • 13.
    Theory. It’s important,valuable. 13 Data In  Analysis  Info Out •Notes •Text •Numeric •Images •Charts/Graphs •Maps •Audio •Video •Atoms  Bits •How? Who?
  • 14.
    Theory. It’s important,valuable. 14 Data In  Analysis  Info Out• Notes • Text • Numeric • Images • Charts/Graphs • Maps • Audio • Video • AtomsBits How? Who? • What are we looking for? How can we be surprised? • Source • Definition • Context • Estimating • Counting • Statistical • Geostatistical • Social Network Analysis • Forensic accounting
  • 15.
    Theory. It’s important,valuable. 15 Data In  Analysis  Info Out• Notes • Text • Numeric • Images • Charts/Graphs • Maps • Audio • Video • AtomsBits How? • What are we looking for? How can we be surprised? • Source • Definition • Context • Estimating • Counting • Statistical • Geostatistical • Social Network Analysis • Forensic accounting • Broadcast • Web • Audio • Video • Text • Data visualization • Maps • Dynamic databases • Archives
  • 16.
    Theory. It’s important,valuable. 16 Data In  Analysis  Info Out • Notes • Text • Numeric • Images • Charts/Graphs • Maps • Audio • Video • AtomsBits How? • What are we looking for? How can we be surprised? • Source • Definition • Context • Estimating • Counting • Statistical • Geostatistical • Social Network Analysis • Forensic accounting • Broadcast • Web • Audio • Video • Text • Data visualization • Maps • Dynamic databases • Archives Contributed content Contributed content & analysis
  • 17.
    Developing Tools forCollaboration • Who is using wikis? Google Docs? Zoho? • Intra-nets • Inter- and intra-nets
  • 18.
    Goals for JournalismEducation • Raise basic admission standards – Applicants already can write; can do basic arithmetic • Develop/improve tools for newsroom and citizen journalism collaboration – Intranets – Wikis – Google Docs – Blogs, social media, Flickr, etc. – Research customization for journalism • Develop tools for story telling
  • 19.
    Community & Collaboration •Citizen Journalism and Democracy in Africa: “Periodismo Ciudadano y Democracia en África”: Un libro en línea de Fackson Banda • UK: The Guardian uses the citizen journalist to great effect in Expenses Scandal • Investigate your MP’s expenses • Oldweather.org Oldweather.org • NYTimes Knowledge Network El New York Times ofrece un curso online de periodismo ciudadano 19
  • 20.
    Online learning; onlinetools • ICFJ Anywhere Global source for high-quality, online journalism courses taught by seasoned professionals. • Google Refine Google Refine is a power tool for working with messy data, cleaning it up, transforming it from one format into another, extending it with web services, and linking it to databases like Freebase. • Many Eyes Site is set up to allow the internet community to upload data, visualize it, and talk about their discoveries with other people. 20
  • 21.
  • 22.
    Storytelling R&D? Innovation? 22 •Potential for “drill-down” dynamics • Reach URLs • Show data for statistical analysis • Ask questions; capture answers
  • 23.
    Tomorrow? • Our jobis to “monitor the centres of power.” -- Amira Haassaid 23
  • 24.
    Tomorrow? • In ademocracy, news media harvest data; become central warehouse of citizen’s data • A “public library” on steroids • Digital resources for in-house journalists • Create “dashboards” to monitor gov’t. • Potential to create specialized, non- journalism “products” • Provide data to app creators
  • 25.
    Tomorrow? • Paying fornews rapidly declining market. • So what can journalists offer? 25
  • 26.
    "Analytic Journalism: Digital Evolutionin the Datasphere" Tom Johnson Managing Director Inst. for Analytic Journalism Santa Fe, New Mexico USA tom@jtjohnson.com
  • 27.
    Theory. It’s important,it’s valuable. 27 Nothing is as valuable as a good theory.

Editor's Notes

  • #5 'How are your jobs different from five years ago?‘ - http://cronkite.asu.edu/mcguireblog/?p=209 McGuire on MediaThat's what Tim McGuire asked his panelists during a discussion on incorporating New Media into the traditional news industry. "Deadlines are dead," said KNXV-TV news director Joe Hengemuehler. "The deadline is now!"
  • #9 Typically and increasingly requires collaboration of people with diverse and high-level skills.
  • #11 Also Portfolio engine: http://www.youtube.com/watch?v=STh0MN02CRg&p=03212D9AD6B7DA85&index=4&playnext=2 Simtable fire model: http://www.kob.com/article/stories/S1623976.shtml?cat=525
  • #20 boosting its reputation for investigative journalism and circulation figures, but the Guardian is determined to have its share of the scandal cake.  Last week the Guardian launched a 'groundbreaking crowd-sourcing exercise' in which they invited its online readers to help filter the myriad of data on MP's expenses, which was recently released by the government.The project is entirely web based.  The documents, numbering at present 457153 pages, have been uploaded onto a specifically designed section of the Guardian's website, facilitated by the use of a new web framework.  Those interested are invited to create an account and analyse a document presented as un-researched.  It is then up to the reader to decide if it is 'interesting', along Guardian guidelines: "food bills, repeated claims for less than £250 (the limit for claims not backed up by a receipt), and rejected claims".  Next, the reader has to copy out individual entries, make observations as to why specific claims require 'further scrutiny' and the push the 'investigate this' button.  The rest is left to the Guardian team.  The project is still developing, and more pages relating to different MPs are due to be uploaded soon.
  • #24 Although South Africa has a widely spread web of about 1,200 public libraries this does not cover the country-wide need for information. Unfortunately only a few of these libraries are represented on the internet. http://www.goethe.de/ins/za/joh/wis/wtl/ofb/enindex.htm SABC - The SABC’s core business is to deliver a variety of high quality programmes and services through television and radio that informs, educates, entertains and supports the public at large. SABC Mandate: http://www.sabc.co.za/wps/wcm/connect/3bb9fc8044341da1a563e7c4173d8502/SABCMANDATEPOLICY.PDF?MOD=AJPERES&CONVERT_TO=url&CACHEID=3bb9fc8044341da1a563e7c4173d8502
  • #26 Although South Africa has a widely spread web of about 1,200 public libraries this does not cover the country-wide need for information. Unfortunately only a few of these libraries are represented on the internet. http://www.goethe.de/ins/za/joh/wis/wtl/ofb/enindex.htm SABC - The SABC’s core business is to deliver a variety of high quality programmes and services through television and radio that informs, educates, entertains and supports the public at large. SABC Mandate: http://www.sabc.co.za/wps/wcm/connect/3bb9fc8044341da1a563e7c4173d8502/SABCMANDATEPOLICY.PDF?MOD=AJPERES&CONVERT_TO=url&CACHEID=3bb9fc8044341da1a563e7c4173d8502