Analytic Journalism: Investing in an Intellectual Portfolio to Secure Journalism's Financial Future Durban, South Africa 11 September 2006

Journalism is… “ The central purpose of journalism is to provide citizens with accurate and reliable information they need to function in a free society.'‘ — Bill Kovach Committee of Concerned Journalists

“ The central purpose of journalism is to provide citizens with accurate and reliable information they need to function in a free society.'‘ — Bill Kovach Committee of Concerned Journalists

What journalists do The job of journalists is "to monitor the centres of power." Israeli journalist Amira Hass

The job of journalists is "to monitor the centres of power." Israeli journalist Amira Hass

Today’s objectives Discuss journalism (and journalists) as a species existing in the Datasphere Review the changes in the information environment Suggest steps journalism can (Should? Must?) take to adapt to those changes if we are to survive Suggest the possible disappearance of the species journalist serioso

Discuss journalism (and journalists) as a species existing in the Datasphere

Review the changes in the information environment

Suggest steps journalism can (Should? Must?) take to adapt to those changes if we are to survive

Suggest the possible disappearance of the species journalist serioso

Analytic thinking/journalism What is it? Where did it come from? How does it work? What are the necessary variables? Frame the appropriate question Find and retrieve appropriate data Use appropriate analytic tools Show what you know with story-appropriate media

What is it?

Where did it come from?

How does it work?

What are the necessary variables?

Frame the appropriate question

Find and retrieve appropriate data

Use appropriate analytic tools

Show what you know with story-appropriate media

The why and the wherefore of Analytic Journalism Dramatic change in info environment Changes in how modern societies relate to data  information What is Analytic Journalism? Objectives of the Institute for Analytic Journalism

Dramatic change in info environment

Changes in how modern societies relate to data  information

What is Analytic Journalism?

Objectives of the Institute for Analytic Journalism

The Changing Datasphere The Datasphere : that conceptual environment where all data exists in all forms, all media Laws Econ Demographics Entertainment News Cultural knowledge All of history

Species in the Datasphere The Datasphere: that conceptual environment where all information-processing species reside Lawyer Economist Insurance adjuster Driver of garbage truck Journalist Teacher Historian Cop

Communications Revolution 1st Era = Symbols Print Paper Ink Writing 2nd Era = 3rd Era = 101010101010101010101010001011100101

How big is the change? 93% of ALL information – audio, video, print – produced worldwide in 2003 was originally in digital format See UC-Berkeley study on Datasphere http://www.sims.berkeley.edu/how-much-info/index.html

93% of ALL information – audio, video, print – produced worldwide in 2003 was originally in digital format

See UC-Berkeley study on Datasphere http://www.sims.berkeley.edu/how-much-info/index.html

Major trends in dynamic Datasphere Definition of literacy expands “ Transparency” Global Governments, NGOs, corporations Data acquisition power shift to individual “ Available Data Index” & “Journalism Info/Utility Index”

Definition of literacy expands

“ Transparency”

Global

Governments, NGOs, corporations

Data acquisition power shift to individual

“ Available Data Index” & “Journalism Info/Utility Index”

ADQ: “Available Data Index” High Low ADI Time Digital Revolution

JIQ: “Journalism Info/Utility Index” JII Avail. Data Index Time High Low Digital Revolution

JIQ: “Journalism Info/Utility Index” High Low High High Low Avail. Data Index Digital Revolution Citizens’ Zone of Contempt, Disregard, Irrelevance, for the press JII

Journalism and pre-Digital Revolution Canon of Ethics Constitutional freedoms/ legal restrictions History of Profession Journalism education: “To serve the profession,” not lead RW-P process R eporting  W riting  Publishing/ P roducing Good at description; far less good at analysis Journalist as “white-hunter poet” (aka “General Assgn. Reporter”) Driven by adrenalin, not intellect Pre-DigitalRev, Traditional Journalism

Canon of Ethics

Constitutional freedoms/ legal restrictions

History of Profession

Journalism education: “To serve the profession,” not lead

RW-P process R eporting  W riting  Publishing/ P roducing

Good at description; far less good at analysis

Journalist as “white-hunter poet” (aka “General Assgn. Reporter”)

Driven by adrenalin, not intellect

Attributes of Digital Revolution Data -- public and private -- in 1s and 0s divorced from physical media Cheap, powerful desktop & communications hardware Cheap, powerful analytic tools Constant, rapid change in Datasphere

Data -- public and private -- in 1s and 0s divorced from physical media

Cheap, powerful desktop & communications hardware

Cheap, powerful analytic tools

Constant, rapid change in Datasphere

Effects of Digital Revolution Fluid, ubiquitous data and results Reproducible data and info without cost. How to add value? New definitions of literacy RRAW-P Process Research  Reporting  Analysis  Writing  Publishing/Producing Need for constant teaching/learning How did journalism respond?

Fluid, ubiquitous data and results

Reproducible data and info without cost. How to add value?

New definitions of literacy

RRAW-P Process Research  Reporting  Analysis  Writing  Publishing/Producing

Need for constant teaching/learning

How did journalism respond?

Journalism and post-Digital Revolution Post-Digital Revolution Changes Various circulation and marketing strategies, e.g. zoning, newspaper-TV ties, etc. Eventually, PCs/I-net in offices & newsroom “ Soft news” Shovelware web sites CAR (Computer-assisted Reporting) 010101001 0011010110101010100101010 1001010101010010101010101 100100101100101010001010 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 010100101010 Pre-DigitalRev, Traditional Journalism

Post-Digital Revolution Changes

Various circulation and marketing strategies, e.g. zoning, newspaper-TV ties, etc.

Eventually, PCs/I-net in offices & newsroom

“ Soft news”

Shovelware web sites

CAR (Computer-assisted Reporting)

Leading-edge journalism in Digital Age CAR (Computer-assisted Reporting) Acquiring or creating digital data sets Learning/utilizing new – for journalism –knowledge/analysis tools, i.e. sprdsheets, DBF, simple stats, GIS to make maps 010101001 0011010110101010100101010 1001010101010010101010101 100100101100101010001010 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 010100101010 Pre-DigitalRev, Traditional Journalism

CAR (Computer-assisted Reporting)

Acquiring or creating digital data sets

Learning/utilizing new – for journalism –knowledge/analysis tools, i.e. sprdsheets, DBF, simple stats, GIS to make maps

Retrieve/adapt methodologies from other disciplines Analytic Journalism in Digital Age Analytic Journalism Anthropology Geography Biology Criminology Accounting Data Mining 010101001 0011010110101010100101010 1001010101010010101010101 100100101100101010001010 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 1001010101010010101010101 010100101010 Pre-DigitalRev, Traditional Journalism

Retrieve/adapt methodologies from other disciplines

AJ “borrowed” methodologies GIS Epidemiology ; Urban planning ; Enviro-sciences ; Crime analysis ; Cyberspace ; Mapping of Concepts Computational Linguistics (King and Lowe’s “Rare Events in International Conflict”) http://gking.harvard.edu/files/infoex.pdf ThemeRiver http://www. pnl . gov / infoviz / infoviz / themeriver . pdf Forensic Accounting http://www. forensicaccounting .com/home.html Visual Statistics & Complex Adaptive Systems - Web Map http://www. webmap .com/ - Epstein, Steinbruner and Parker. “Modeling Civil Violence: An Agent-Based Computational Approach” http://www.brook.edu/es/dynamics/papers/cviolence/cviolence.pdf

GIS

Epidemiology ; Urban planning ; Enviro-sciences ; Crime analysis ; Cyberspace ; Mapping of Concepts

Computational Linguistics

(King and Lowe’s “Rare Events in International Conflict”) http://gking.harvard.edu/files/infoex.pdf

ThemeRiver http://www. pnl . gov / infoviz / infoviz / themeriver . pdf

Forensic Accounting http://www. forensicaccounting .com/home.html

Visual Statistics & Complex Adaptive Systems - Web Map http://www. webmap .com/ - Epstein, Steinbruner and Parker. “Modeling Civil Violence: An Agent-Based Computational Approach” http://www.brook.edu/es/dynamics/papers/cviolence/cviolence.pdf

Focus always international Retrieve/adapt methodologies from other disciplines IAJ: hinge between other disciplines/ methodologies and journalism Adopt new analytic tools appropriate for journalism, e.g. AskSam , Cartagio , ArcGIS Focus on making data – broadly defined – available for user customization and analysis Analytic Journalism in Digital Age

Focus always international

Retrieve/adapt methodologies from other disciplines

IAJ: hinge between other disciplines/ methodologies and journalism

Adopt new analytic tools appropriate for journalism, e.g. AskSam , Cartagio , ArcGIS

Focus on making data – broadly defined – available for user customization and analysis

Hunter College-NYTimes satellite maps of WTC disaster

John Snow’s aledged cholera map- London - 1854 John Snow’s alleged cholera map London - 1854

Urban Planning Sat. Pics-Lotlines Urban Planning Tax assessments Public utilities/services Traffic Planned growth? Sprawl? Educational planning

Urban Planning

Tax assessments

Public utilities/services

Traffic

Planned growth? Sprawl?

Educational planning

Environmental Planning – watershed map English River Watershed Study by students in Mid-Prairie Middle School Kalona, Iowa

Portland, Oregon Crime mapping Note how database produces graph and map. Portland, Oregon site at http://www. portlandpolicebureau .com/ crimemapper .html

Conceptual mapping of cyberspace VR visualization of Web traffic

PowerPlayers1 Mapping power relationships http://theyrule.orgo.org/

“ ThemeRiver” computational linguistics and resulting visualization of news story themes. “ Theme River” computational linguistics and resulting visualization of news story topics. Curves indicate “tension index” over time Source: http://www. pnl . gov / infoviz /graphics.html

Online enterprise Dynamic system model Source: http://www. simventure .com/models.html

Data mining – advanced Business Intelligence software http://www.pcmag.com/article/0,2997,s%253D1738%2526a%253D7640,00.asp

Database software – online and shrinkwrapped http://www.pcmag.com/article/0,2997,s%3D400%26a%3D14601,00.asp

Some Analytic Journalism teaching materials Statistics - http://people.bu.edu/ssr3/stat2.htm E-mail tracing - http://people.bu.edu/ssr3/tracing.htm Middleberg/Ross survey - http://people.bu.edu/ssr3/cyber7.htm Covering the Census, with GIS - http://www.columbia.edu/~ssr3/census.htm Basic numbers skills - http://www.columbia.edu/~ssr3/NUMBER12.htm Evaluating what you find - http://www.columbia.edu/~ssr3/eval.htm Calculator for confidence levels in polls - http://www.columbia.edu/~ssr3/confross.htm

Statistics - http://people.bu.edu/ssr3/stat2.htm

E-mail tracing - http://people.bu.edu/ssr3/tracing.htm

Middleberg/Ross survey - http://people.bu.edu/ssr3/cyber7.htm

Covering the Census, with GIS - http://www.columbia.edu/~ssr3/census.htm

Basic numbers skills - http://www.columbia.edu/~ssr3/NUMBER12.htm

Evaluating what you find - http://www.columbia.edu/~ssr3/eval.htm

Calculator for confidence levels in polls - http://www.columbia.edu/~ssr3/confross.htm

Inst. for Analytic Journalism General Systems Theory Statistics Visual Statistics and Infographics Simulation modeling Our “Cornerstones”

General Systems Theory

Statistics

Visual Statistics and Infographics

Simulation modeling

Analytic Journalism: Investing in an Intellectual Portfolio to Secure Journalism's Financial Future Durban, South Africa 11 September 2006