OKCON 2011
OPEN DATA IN THE NEWSROOM
WHAT‘S THE
STORY?
Mirko Lorenz • Information Architect/Journalist • Deutsche Welle/Innovation Projects


June 30, 2011 • Kalkscheune Berlin
THIS IS ALL

 BETA
needs work
WELCOME.
IF YOU ARE NEW, THIS IS THE CONTEXT
This is a presentation from at OK Con 2011 in Berlin.
The topic is „data-driven journalism“ (DDJ).

Our idea of data-driven journalism is to view the analysis of
data as a rich source for good stories.

Data-driven journalism is inclusive, meaning that other forms of
journalism (investigative, computer-assisted reporting) are
surely part of the broader quest to find ways for relevant,
insightful journalism in a world driven by data.

Plus, we hope that by adding data to the newsroom there will
be new models to finance journalism. And why not, if the story
is good.

Data-driven journalism @ Wikipedia:
http://en.wikipedia.org/wiki/Data_driven_journalism
ME: TURNED
JOURNALIST
INFORMATION ARCHITECT
FOCUS: TURNING IDEAS INTO
PLATFORMS, WORKFLOWS, STORIES
WORKING@DEUTSCHE WELLE
INNOVATION PROJECTS
JOURNALISM TRAINER
DATA IN THE NEWSROOM?
DATA IN THE NEWSROOM?


NOW: OPEN DATA IS A WINNER
DATA IN THE NEWSROOM?


NOW: OPEN DATA IS A WINNER

NEXT: CHALLENGES AHEAD
DATA IN THE NEWSROOM?


NOW: OPEN DATA IS A WINNER

NEXT: CHALLENGES AHEAD

FUTURE: PATTERN CHANGE?
#1:
OPEN DATA IS A WINNER
WE ALL
NEED THE
DATA
THE IDEA ACCEPTED
  ON HIGH LEVELS
DATA INSIGHTS FOR EVERYONE?


                        WORLD
                 COUNTRY
PEOPLE        CITY
         GROUP
    OPEN + NOT-SO-OPEN DATA
DATA INSIGHTS FOR EVERYONE?
    MEDIA
                        WORLD
                 COUNTRY
PEOPLE        CITY
         GROUP
    OPEN + NOT-SO-OPEN DATA
#2:
DATA-DRIVEN JOURNALISM:
     WHY BOTHER?
OBSERVATIONS:
MUCH PROGRESS LAST YEAR
SIMON ROGERS IS AN OUTLIER
NEED TO TRAIN JOURNALISTS
WHAT ARE LONG-TERM GOALS?
ON A GLOWING RECTANGLE NEAR YOU
THE
MOVIE?   DATA STORY
          DIGGING DEEPER
                                              „Surprisingly
                                                helpful“
                                                  -- User




             „Our city is now a better place“
                         -- Local Authority
         „It‘s a wonder how we could do without it“
                          -- Government

               AN OPEN DATA PRODUCTION
CHALLENGES:
	 DIRTY DATA
  DATA NEEDS STORY
  CREATING A NEW „CAMERA“
HOW DIRTY IS THE DATA?




Tar Sands of Alberta. Image: Some rights reserved by NWFblogs
HOW DIRTY IS THE DATA?


                     EXTRACTING MEANING FROM DATA WILL BE HARD WORK
                   WON‘T COME AT ZERO-COST OR WITHOUT UNWANTED EFFECTS




Tar Sands of Alberta. Image: Some rights reserved by NWFblogs
NOT ALL DATA-SETS CREATED EQUAL
             INCOMPLETE     WRONG
                            FOCUS COMPETITION
   WRONG
   FORMAT
                                      CREATED FOR
                   DIRTY
                     Text              MARKETING,
  MISTAKES         DATA               NOT INSIGHTS
   WHEN
 COLLECTED                         INTENTIONALLY
                                      FORGED
            INSIGHTS KEPT SECRET
Journalists,
where do you
add value?
      - Jeff Jarvis
DATA + STORY = VALUE TO PUBLIC
DATA + STORY: EXAMPLE




         Source: Information is Beautiful, by David McCandless
A „NEW CAMERA“ TO VIEW THE WORLD




           Source: Information is Beautiful, by David McCandless,
       quoted from talk of David in Brussels @ Visualizing Europe, 2011
A „NEW CAMERA“ TO VIEW THE WORLD




           Source: Information is Beautiful, by David McCandless,
       quoted from talk of David in Brussels @ Visualizing Europe, 2011
DATA + STORY
„NEVER COMING HOME“




 Source: http://mediastorm.com//publication/never-coming-home
#3:
WHAT‘S KEEPING US?
BARRIERS FOR DATA
   JOURNALISM
REBUILDING THE NEWSROOM -
WHY AND HOW?
TODAY‘S MEDIA MODELS
 Print   Web   Blogs *
                           iPad Mobile




               Most media companies
               still think of data and
               digital formats as an
               extension, not as the
               foundation.

               *Blogs are red, because most media did not get the idea
FROM SINGLE STORIES...
...TO DATA-BASED CONTEXT
DATA AS A FOUNDATION
Database    Web           Print
           Overview
                           Article
                                     Depth
            Articles
 Crime       Maps
                          Dossier    Competency
                        Background
 Data      Statistics
                          Context    Relevance
           Analysis
             Audio                   Trustability
            Photo
             Video
SITUATION OF JOURNALISTS
MOST JOURNALIST‘S SITUATION
                  NO ACCESS TO
     FAILING
                     SERVER
     MODELS
                                 NO SUPPORT
 RISING           NO
                  Text
WORKLOAD
               TRAINING           TOO MANY
                                 DATA TOOLS
    TIME
  PRESSURE        OLD-SCHOOL
                     CMS
STRANGE TIMES:
LAY-OFFS IN NEWSROOMS VS.
TALENT SHORTAGE IN DIGITAL
BUSINESS
#4:
DEVELOPING THE FIELD
WHAT ARE THE TRAINING NEEDS?
WHAT WOULD BE A GOOD WORKFLOW?

      A workflow for video and data journalism using next-gen cloud storage



          Team       Ingestion      Filter     Produce     Publish     Distribute   Popularity




     Goals
     Teamwork vs. one-man-show
     Enabling use of data in newsrooms
     Workflow as driver of value, better integration
      Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu




    Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
WHAT WOULD BE A GOOD WORKFLOW?

     VISION Cloud Media Use Case
      A workflow for video and data journalism using next-gen cloud storage



          Team       Ingestion      Filter     Produce     Publish     Distribute   Popularity




     Goals
     Teamwork vs. one-man-show
     Enabling use of data in newsrooms
     Workflow as driver of value, better integration
      Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu




    Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
Text
WHAT ARE THE STEPS?



Team   Ingest      Filter                   Produce                       Publish         Distribute            Measure




                Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu

                Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
SMART INTERACTION
               Scraperwiki      Open Calais      Video editing    Data Visualization       Tools         Story Presentation        Multi-platform                  Collecting
 STORLETS      Needlebase      Google Refine       Final Cut          R Studio             Storify           Wordpress               Distribution           usage data over time,
                                                                      Protovis         Document Cloud        Memolane              File Conversion                 based on
 & APIs                                                                                                       Fullstory                  Flash                 files, not pages
                                                                     Gapminder
                                                                                                         Other CMS in use               HTML5
                                                                                                                                        iTunes
                                VIDEO/DATA
               VIDEO/DATA                                    PRODUCTION WITH TEAMS                                                       iPod
                                 FILTERING
               INGESTION                                                                                                               Android
                                METADATA
   SET-UP                                                                                                                                Nokia
    TEAM                                                                                                                                  RIM
PROJECT NAME




    Team          Ingestion    Filter/Metadata              Produce video oder data story                     Publish         Multi-platform distribution         Popularity




                                                        FIND SNIPPETS OF VIDEO/DATA FOR RE-USE



                                ING    F
                           RAIS TION O
                               C                                                                                              NAL            DAT
  STORAGE                   TRA     E    )                                                                               ATIO                    A MO
                        ABS TORAG SULES                  ADVANCED CAPABILITIES
                                                                                 FOR CLOUD-BASED STOR
                                                                                                                      PUT GE
                                                                                                                   COM ORA                      FED BILITY A
   ACCESS                   S
                                AC AP                                                                   AGE           ST                            ERA
                                                                                                                                                        TION ND
   RIGHTS                    DAT
                      (M ETA
  SECURITY




 CLOUD                                                 PURE STORAGE LAYER - KEEPING FILES AND VERSIONS OF FILES OVER TIME




                              Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
#5:
PATTERN CHANGE
Q: PATTERN CHANGE?
 A: LOOKING INTO THE DATA,
  USING WHAT WE FIND FOR
       GOOD PURPOSES
CAN RESULT IN A BIG SHIFT OF
           POWER
QUIZ:
WHAT ARE AR THE INSIGHTS &
TOOLS FUTURE GENERATIONS
 WILL THINK ARE OBVIOUS?
WHAT CAN BE DONE? (MOST EXAMPLES ON THIS PAGE ARE LINKED)

SURPRISE




CLARITY




TRUST
#HOPE:
MEDIA COMPANIES BECOMING
TRUSTED DATA HUBS
THE FUTURE: INSTANT CLARITY BEFORE WE MAKE DECISIONS




                                         Source: Visua.ly
THE FUTURE: INSTANT CLARITY BEFORE WE MAKE DECISIONS




                                         Source: Visua.ly
THE FUTURE: INSTANT CLARITY BEFORE WE MAKE DECISIONS




                                         Source: Visua.ly
FINALLY: THE CALL TO ACTION
(1) LET‘S WORK TOGETHER
(2) DEVELOPERS/JOURNALISTS/VISUALIZERS
(3) SMALL TEAMS CAN ROCK THIS PLACE
Caution:
 There’s no standard
career path to become
  a deconstructor of
      wrongness
Quote from David H. Freedmann, „Wrong“, Review of Book http://www.nytimes.com/2010/06/11/books/excerpt-wrong.html
THANK YOU!
Brian Storm     Simon Rogers     Walter Mossberg      Rafat Ali    Ariana Huffington




Aron Pilhofer    Jeff Jarvis    Nicolas Kayser-Bril Michelle Minkoff Mercedes Bunz




Mathias Eberl   Lorenz Matzat      Tracy Boyer     Paul Bradshaw    Amanda Cox
HOUSEKEEPING: HOW TO CONNECT

Twitter: #ddj
Please use this hashtag for „data-driven journalism“ - it‘s nicely short

Survey:
The survey to find out about training needs for data journalists is still open
(as of July 2011)
http://www.surveymonkey.com/s/data_journalism

Data-driven journalism group @ EJC
+150 people discussing and developing.
http://community.ejc.net/group/datadrivenjournalism

Data-driven journalism website
http://datadrivenjournalism.net/

My website:
http://www.mirkolorenz.com
IDEAS, DISCUSSION, ARGUE?
TWITTER: #DDJ
@MIRKOLORENZ
Some ideas and concepts presented here have received funding from the
European Community's Seventh FrameworkProgramme (FP7/2007-2013)
under grant agreement n° 257019 (VISION Cloud)

Open Data in the Newsroom: What's the story? (Talk from OK Con 2011 in Berlin)

  • 1.
    OKCON 2011 OPEN DATAIN THE NEWSROOM WHAT‘S THE STORY? Mirko Lorenz • Information Architect/Journalist • Deutsche Welle/Innovation Projects June 30, 2011 • Kalkscheune Berlin
  • 2.
    THIS IS ALL BETA needs work
  • 3.
    WELCOME. IF YOU ARENEW, THIS IS THE CONTEXT This is a presentation from at OK Con 2011 in Berlin. The topic is „data-driven journalism“ (DDJ). Our idea of data-driven journalism is to view the analysis of data as a rich source for good stories. Data-driven journalism is inclusive, meaning that other forms of journalism (investigative, computer-assisted reporting) are surely part of the broader quest to find ways for relevant, insightful journalism in a world driven by data. Plus, we hope that by adding data to the newsroom there will be new models to finance journalism. And why not, if the story is good. Data-driven journalism @ Wikipedia: http://en.wikipedia.org/wiki/Data_driven_journalism
  • 4.
    ME: TURNED JOURNALIST INFORMATION ARCHITECT FOCUS:TURNING IDEAS INTO PLATFORMS, WORKFLOWS, STORIES WORKING@DEUTSCHE WELLE INNOVATION PROJECTS JOURNALISM TRAINER
  • 5.
    DATA IN THENEWSROOM?
  • 6.
    DATA IN THENEWSROOM? NOW: OPEN DATA IS A WINNER
  • 7.
    DATA IN THENEWSROOM? NOW: OPEN DATA IS A WINNER NEXT: CHALLENGES AHEAD
  • 8.
    DATA IN THENEWSROOM? NOW: OPEN DATA IS A WINNER NEXT: CHALLENGES AHEAD FUTURE: PATTERN CHANGE?
  • 9.
  • 10.
  • 11.
    THE IDEA ACCEPTED ON HIGH LEVELS
  • 12.
    DATA INSIGHTS FOREVERYONE? WORLD COUNTRY PEOPLE CITY GROUP OPEN + NOT-SO-OPEN DATA
  • 13.
    DATA INSIGHTS FOREVERYONE? MEDIA WORLD COUNTRY PEOPLE CITY GROUP OPEN + NOT-SO-OPEN DATA
  • 14.
  • 15.
    OBSERVATIONS: MUCH PROGRESS LASTYEAR SIMON ROGERS IS AN OUTLIER NEED TO TRAIN JOURNALISTS WHAT ARE LONG-TERM GOALS?
  • 16.
    ON A GLOWINGRECTANGLE NEAR YOU THE MOVIE? DATA STORY DIGGING DEEPER „Surprisingly helpful“ -- User „Our city is now a better place“ -- Local Authority „It‘s a wonder how we could do without it“ -- Government AN OPEN DATA PRODUCTION
  • 17.
    CHALLENGES: DIRTY DATA DATA NEEDS STORY CREATING A NEW „CAMERA“
  • 18.
    HOW DIRTY ISTHE DATA? Tar Sands of Alberta. Image: Some rights reserved by NWFblogs
  • 19.
    HOW DIRTY ISTHE DATA? EXTRACTING MEANING FROM DATA WILL BE HARD WORK WON‘T COME AT ZERO-COST OR WITHOUT UNWANTED EFFECTS Tar Sands of Alberta. Image: Some rights reserved by NWFblogs
  • 20.
    NOT ALL DATA-SETSCREATED EQUAL INCOMPLETE WRONG FOCUS COMPETITION WRONG FORMAT CREATED FOR DIRTY Text MARKETING, MISTAKES DATA NOT INSIGHTS WHEN COLLECTED INTENTIONALLY FORGED INSIGHTS KEPT SECRET
  • 21.
    Journalists, where do you addvalue? - Jeff Jarvis
  • 22.
    DATA + STORY= VALUE TO PUBLIC
  • 23.
    DATA + STORY:EXAMPLE Source: Information is Beautiful, by David McCandless
  • 24.
    A „NEW CAMERA“TO VIEW THE WORLD Source: Information is Beautiful, by David McCandless, quoted from talk of David in Brussels @ Visualizing Europe, 2011
  • 25.
    A „NEW CAMERA“TO VIEW THE WORLD Source: Information is Beautiful, by David McCandless, quoted from talk of David in Brussels @ Visualizing Europe, 2011
  • 26.
    DATA + STORY „NEVERCOMING HOME“ Source: http://mediastorm.com//publication/never-coming-home
  • 27.
  • 28.
  • 29.
    TODAY‘S MEDIA MODELS Print Web Blogs * iPad Mobile Most media companies still think of data and digital formats as an extension, not as the foundation. *Blogs are red, because most media did not get the idea
  • 30.
  • 31.
  • 32.
    DATA AS AFOUNDATION Database Web Print Overview Article Depth Articles Crime Maps Dossier Competency Background Data Statistics Context Relevance Analysis Audio Trustability Photo Video
  • 33.
  • 34.
    MOST JOURNALIST‘S SITUATION NO ACCESS TO FAILING SERVER MODELS NO SUPPORT RISING NO Text WORKLOAD TRAINING TOO MANY DATA TOOLS TIME PRESSURE OLD-SCHOOL CMS
  • 35.
    STRANGE TIMES: LAY-OFFS INNEWSROOMS VS. TALENT SHORTAGE IN DIGITAL BUSINESS
  • 36.
  • 37.
    WHAT ARE THETRAINING NEEDS?
  • 38.
    WHAT WOULD BEA GOOD WORKFLOW? A workflow for video and data journalism using next-gen cloud storage Team Ingestion Filter Produce Publish Distribute Popularity Goals Teamwork vs. one-man-show Enabling use of data in newsrooms Workflow as driver of value, better integration Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
  • 39.
    WHAT WOULD BEA GOOD WORKFLOW? VISION Cloud Media Use Case A workflow for video and data journalism using next-gen cloud storage Team Ingestion Filter Produce Publish Distribute Popularity Goals Teamwork vs. one-man-show Enabling use of data in newsrooms Workflow as driver of value, better integration Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
  • 40.
    Text WHAT ARE THESTEPS? Team Ingest Filter Produce Publish Distribute Measure Note: Deutsche Welle is a partner in this EU-project. Website here: http://www.visioncloud.eu Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
  • 41.
    SMART INTERACTION Scraperwiki Open Calais Video editing Data Visualization Tools Story Presentation Multi-platform Collecting STORLETS Needlebase Google Refine Final Cut R Studio Storify Wordpress Distribution usage data over time, Protovis Document Cloud Memolane File Conversion based on & APIs Fullstory Flash files, not pages Gapminder Other CMS in use HTML5 iTunes VIDEO/DATA VIDEO/DATA PRODUCTION WITH TEAMS iPod FILTERING INGESTION Android METADATA SET-UP Nokia TEAM RIM PROJECT NAME Team Ingestion Filter/Metadata Produce video oder data story Publish Multi-platform distribution Popularity FIND SNIPPETS OF VIDEO/DATA FOR RE-USE ING F RAIS TION O C NAL DAT STORAGE TRA E ) ATIO A MO ABS TORAG SULES ADVANCED CAPABILITIES FOR CLOUD-BASED STOR PUT GE COM ORA FED BILITY A ACCESS S AC AP AGE ST ERA TION ND RIGHTS DAT (M ETA SECURITY CLOUD PURE STORAGE LAYER - KEEPING FILES AND VERSIONS OF FILES OVER TIME Visualization inspired by „A Web Site Designed“, created by John Furness of Simple Square
  • 42.
  • 43.
    Q: PATTERN CHANGE? A: LOOKING INTO THE DATA, USING WHAT WE FIND FOR GOOD PURPOSES CAN RESULT IN A BIG SHIFT OF POWER
  • 44.
    QUIZ: WHAT ARE ARTHE INSIGHTS & TOOLS FUTURE GENERATIONS WILL THINK ARE OBVIOUS?
  • 45.
    WHAT CAN BEDONE? (MOST EXAMPLES ON THIS PAGE ARE LINKED) SURPRISE CLARITY TRUST
  • 46.
  • 47.
    THE FUTURE: INSTANTCLARITY BEFORE WE MAKE DECISIONS Source: Visua.ly
  • 48.
    THE FUTURE: INSTANTCLARITY BEFORE WE MAKE DECISIONS Source: Visua.ly
  • 49.
    THE FUTURE: INSTANTCLARITY BEFORE WE MAKE DECISIONS Source: Visua.ly
  • 50.
    FINALLY: THE CALLTO ACTION (1) LET‘S WORK TOGETHER (2) DEVELOPERS/JOURNALISTS/VISUALIZERS (3) SMALL TEAMS CAN ROCK THIS PLACE
  • 51.
    Caution: There’s nostandard career path to become a deconstructor of wrongness Quote from David H. Freedmann, „Wrong“, Review of Book http://www.nytimes.com/2010/06/11/books/excerpt-wrong.html
  • 52.
    THANK YOU! Brian Storm Simon Rogers Walter Mossberg Rafat Ali Ariana Huffington Aron Pilhofer Jeff Jarvis Nicolas Kayser-Bril Michelle Minkoff Mercedes Bunz Mathias Eberl Lorenz Matzat Tracy Boyer Paul Bradshaw Amanda Cox
  • 53.
    HOUSEKEEPING: HOW TOCONNECT Twitter: #ddj Please use this hashtag for „data-driven journalism“ - it‘s nicely short Survey: The survey to find out about training needs for data journalists is still open (as of July 2011) http://www.surveymonkey.com/s/data_journalism Data-driven journalism group @ EJC +150 people discussing and developing. http://community.ejc.net/group/datadrivenjournalism Data-driven journalism website http://datadrivenjournalism.net/ My website: http://www.mirkolorenz.com
  • 54.
    IDEAS, DISCUSSION, ARGUE? TWITTER:#DDJ @MIRKOLORENZ Some ideas and concepts presented here have received funding from the European Community's Seventh FrameworkProgramme (FP7/2007-2013) under grant agreement n° 257019 (VISION Cloud)