Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
NewsDiffs: Version Controlling the News                                      Eric Price           Margaret Sullivan        ...
NewsDiffs          Online news is different from print.Eric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controll...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.Eric Price, M...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffs          Online news is different from print.                 Print: hard to change, daily deadlines.             ...
NewsDiffsEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   3 / 30
NewsDiffsEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   3 / 30
Outline of Talk  1    Motivation and Creation  2    Case Studies  3    Future  4    Q&AEric Price, Margaret Sullivan (MIT,...
Outline of Talk  1    Motivation and Creation  2    Case Studies  3    Future  4    Q&AEric Price, Margaret Sullivan (MIT,...
Occupy Wall Street arrests              After allowing them          onto the bridge, police cut          off and arrested ...
N’kisi the telepathic parrot Found via Language Log              N’kisi’s remarkable abilities, which are          said to...
N’kisi the telepathic parrot Found via Language Log              N’kisi’s remarkable abilities, which are          said to...
N’kisi the telepathic parrot Found via Language Log              N’kisi’s remarkable abilities, which are          said to...
N’kisi the telepathic parrot Found via Language Log              N’kisi’s remarkable abilities, which are          said to...
The public editor, a year before NewsDiffs             Right now, tracking changes is not a priority at The Times.         ...
NewsDiffs team          Jennifer 8. Lee                            Greg Price                     Eric PriceEric Price, Mar...
Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013...
Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013...
Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013...
Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013...
Knight-Mozilla Open News Hackathon                            27 hours of furious codingEric Price, Margaret Sullivan (MIT...
Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013...
A permanent record is feasible          Recall The Times’s statement:              [I]t’s unrealistic to preserve an “immu...
Technical overview                                                                     Scraper                www.nytimes....
*Not quite one weekend          Another day of work after each of 3, 10, 22 weeks.          Scaling issues                ...
PressEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   14 / 30
Press              [A] more comprehensive archive that retains all significant          versions of an article (and all cor...
PressEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   16 / 30
PressEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   16 / 30
PressEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-11   16 / 30
Easy to extend          We’re tracking the New York Times, CNN, BBC, Politico.          To track another site, need to wri...
NewsDiffs is Free Software          Forks:                                           http://redactado.com.ar/Eric Price, Ma...
NewsDiffs is Free Software          Forks:                                           http://redactado.com.ar/Eric Price, Ma...
NewsDiffs is Free Software          Forks:                                               http://newsdiffs.es/Eric Price, Ma...
NewsDiffs is Free Software          Forks:                                               http://newsdiffs.es/          Patc...
Statistics          Tracking 28000 NYT articles (62000 over all sources).          44% of articles changed at least once. ...
Outline of Talk  1    Motivation and Creation  2    Case Studies  3    Future  4    Q&AEric Price, Margaret Sullivan (MIT,...
Examples: Nuclear Talks with IranEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-...
Examples: Sandy Hook ShootingEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-1...
Examples: Edward Koch ObituaryEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-...
Examples: Romney and BenghaziEric Price, Margaret Sullivan (MIT, NYT)   NewsDiffs: Version Controlling the News   2013-03-1...
Examples: Romney and Benghazi  In first version:              For a country looking to understand how Mr. Romney, a        ...
Outline of Talk  1    Motivation and Creation  2    Case Studies  3    Future  4    Q&AEric Price, Margaret Sullivan (MIT,...
Goals          Goals for NewsDiffs             1   Reference for known interesting changes.             2   Unearth interes...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Simple example of study That vs. Who          The council could choose a pope that all factions would recognize.          ...
Conclusions          News websites should keep a public record of what they publish.          In the meantime, NewsDiffs fil...
Outline of Talk  1    Motivation and Creation  2    Case Studies  3    Future  4    Q&AEric Price, Margaret Sullivan (MIT,...
Upcoming SlideShare
Loading in …5
×

Version Controlling the News. How We Can Archive. SXSW Presentation on NewsDiffs

3,799 views

Published on

Version Controlling the News. How We Can Archive. SXSW Presentation on NewsDiffs. A look at how version control software can track the changes of online articles in The New York Times, Politico, BBC and CNN. Eric Price and Margaret Sullivan, Public Editor of The New York Times, presented at SXSW 2013. http://schedule.sxsw.com/2013/events/event_FP990508

Published in: News & Politics
  • Be the first to comment

  • Be the first to like this

Version Controlling the News. How We Can Archive. SXSW Presentation on NewsDiffs

  1. 1. NewsDiffs: Version Controlling the News Eric Price Margaret Sullivan MIT The New York Times 2013-03-11 http://newsdiffs.org/Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 1 / 30
  2. 2. NewsDiffs Online news is different from print.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  3. 3. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  4. 4. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  5. 5. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle:Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  6. 6. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  7. 7. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story. Editor makes a pass or two.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  8. 8. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story. Editor makes a pass or two. (Another) reporter rewrites the story.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  9. 9. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story. Editor makes a pass or two. (Another) reporter rewrites the story. Editor makes another pass or two.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  10. 10. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story. Editor makes a pass or two. (Another) reporter rewrites the story. Editor makes another pass or two. Libraries archive print version, not what people actually read.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  11. 11. NewsDiffs Online news is different from print. Print: hard to change, daily deadlines. Online: easy to change, deadline now. Online news articles have a lifecycle: Reporter writes a rushed story. Editor makes a pass or two. (Another) reporter rewrites the story. Editor makes another pass or two. Libraries archive print version, not what people actually read. NewsDiffs tracks stories as they evolve.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 2 / 30
  12. 12. NewsDiffsEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 3 / 30
  13. 13. NewsDiffsEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 3 / 30
  14. 14. Outline of Talk 1 Motivation and Creation 2 Case Studies 3 Future 4 Q&AEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 4 / 30
  15. 15. Outline of Talk 1 Motivation and Creation 2 Case Studies 3 Future 4 Q&AEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 5 / 30
  16. 16. Occupy Wall Street arrests After allowing them onto the bridge, police cut off and arrested dozens of occupy wall street demon- strators. Lede rewritten to remove first bit. Lucky someone must have kept the old tab open! Reporter’s defense: body of article consistent. Hard to judge without access to old version.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 6 / 30
  17. 17. N’kisi the telepathic parrot Found via Language Log N’kisi’s remarkable abilities, which are said to include telepathy, feature in the latest BBC Wildlife Magazine. 2004: BBC Science article appearsEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 7 / 30
  18. 18. N’kisi the telepathic parrot Found via Language Log N’kisi’s remarkable abilities, which are said to include telepathy, feature in the latest BBC Wildlife Magazine. 2004: BBC Science article appears 2006: “Telepathy” removed; no correctionEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 7 / 30
  19. 19. N’kisi the telepathic parrot Found via Language Log N’kisi’s remarkable abilities, which are said to include telepathy, feature in the latest BBC Wildlife Magazine. 2004: BBC Science article appears 2006: “Telepathy” removed; no correction 2007 (May): Article completely replacedEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 7 / 30
  20. 20. N’kisi the telepathic parrot Found via Language Log N’kisi’s remarkable abilities, which are said to include telepathy, feature in the latest BBC Wildlife Magazine. 2004: BBC Science article appears 2006: “Telepathy” removed; no correction 2007 (May): Article completely replaced 2007 (August): “Correction” appears: Note: This story about animal communication has replaced an earlier one on this page which contained factual inaccuracies we were unable to correct. As a result, the original story is no longer in our archive. It is still visible elsewhere, via [link to WayBack Machine].Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 7 / 30
  21. 21. The public editor, a year before NewsDiffs Right now, tracking changes is not a priority at The Times. As [the new executive editor Jill Abramson] told me, it’s unrealistic to preserve an “immutable, permanent record of everything we have done.”Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 8 / 30
  22. 22. NewsDiffs team Jennifer 8. Lee Greg Price Eric PriceEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 9 / 30
  23. 23. Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  24. 24. Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  25. 25. Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  26. 26. Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  27. 27. Knight-Mozilla Open News Hackathon 27 hours of furious codingEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  28. 28. Knight-Mozilla Open News HackathonEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 10 / 30
  29. 29. A permanent record is feasible Recall The Times’s statement: [I]t’s unrealistic to preserve an “immutable, permanent record of everything we have done.” Wikipedia does it. Version control is a solved problem. We did it in one* weekend, from the outside.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 11 / 30
  30. 30. Technical overview Scraper www.nytimes.com BeautifulSoup parser MySQL Database of Article URLs Git repository nytimes.com/2013/...ating.html of text of all articles nytimes.com/2013/...-jail.html Website You Django Google diff-match-patchEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 12 / 30
  31. 31. *Not quite one weekend Another day of work after each of 3, 10, 22 weeks. Scaling issues Running on AFS, a networked file system Moved version metadata from git to MySQL. Optimized queries to both backends UI improvements.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 13 / 30
  32. 32. PressEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 14 / 30
  33. 33. Press [A] more comprehensive archive that retains all significant versions of an article (and all corrections) would send readers a strong message that The Times is committed to full transparency and accountability. [...] As NewsDiffs demonstrates, if you don’t make yourself accountable nowadays, someone else will do it for you.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 15 / 30
  34. 34. PressEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 16 / 30
  35. 35. PressEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 16 / 30
  36. 36. PressEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 16 / 30
  37. 37. Easy to extend We’re tracking the New York Times, CNN, BBC, Politico. To track another site, need to write code to extract plain text from webpage. 30-40 lines of code; takes maybe one hour. But resource constraints: running on free MIT servers out of my account.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 17 / 30
  38. 38. NewsDiffs is Free Software Forks: http://redactado.com.ar/Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 18 / 30
  39. 39. NewsDiffs is Free Software Forks: http://redactado.com.ar/Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 18 / 30
  40. 40. NewsDiffs is Free Software Forks: http://newsdiffs.es/Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 18 / 30
  41. 41. NewsDiffs is Free Software Forks: http://newsdiffs.es/ Patches: Received (and merged) patch to parse tagesschau.de.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 18 / 30
  42. 42. Statistics Tracking 28000 NYT articles (62000 over all sources). 44% of articles changed at least once. 20-30% in opinion, books, fashion sections 55-60% in sports, NY region, world sections 15% of articles changed at least three times. 9% have official corrections. 4% have byline changes. 11% in world section.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 19 / 30
  43. 43. Outline of Talk 1 Motivation and Creation 2 Case Studies 3 Future 4 Q&AEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 20 / 30
  44. 44. Examples: Nuclear Talks with IranEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 21 / 30
  45. 45. Examples: Sandy Hook ShootingEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 22 / 30
  46. 46. Examples: Edward Koch ObituaryEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 23 / 30
  47. 47. Examples: Romney and BenghaziEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 24 / 30
  48. 48. Examples: Romney and Benghazi In first version: For a country looking to understand how Mr. Romney, a Republican candidate with no foreign policy experience, would respond to a major crisis, this was a first glimpse. And as an adviser to the campaign who worked in the George W. Bush administration said on Wednesday, Mr. Romney’s accusation [...] looked like “he had forgotten the first rule in a crisis: don’t start talking before you understand what’s happening.”Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 25 / 30
  49. 49. Outline of Talk 1 Motivation and Creation 2 Case Studies 3 Future 4 Q&AEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 26 / 30
  50. 50. Goals Goals for NewsDiffs 1 Reference for known interesting changes. 2 Unearth interesting changes. 3 Study the process of journalistic editing. Currently only satisfying (1) well. To satisfy the others, need Automated tools to sift through the changes for interesting ones. Someone to use our data for researchEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 27 / 30
  51. 51. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  52. 52. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  53. 53. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  54. 54. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people. When do reporters make mistakes that editors catch?Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  55. 55. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people. When do reporters make mistakes that editors catch? Mitik, the baby walrus who was orphanedEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  56. 56. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people. When do reporters make mistakes that editors catch? Mitik, the baby walrus who was orphaned The giant panda cub who diedEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  57. 57. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people. When do reporters make mistakes that editors catch? Mitik, the baby walrus who was orphaned The giant panda cub who died The subatomic analogue of cats who are alive and dead at the same timeEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  58. 58. Simple example of study That vs. Who The council could choose a pope that all factions would recognize. The council could choose a pope whom all factions would recognize. Rule: “who” refers to people, “that” to non-people. When do reporters make mistakes that editors catch? Mitik, the baby walrus that was orphaned The giant panda cub that died The subatomic analogue of cats that are alive and dead at the same timeEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 28 / 30
  59. 59. Conclusions News websites should keep a public record of what they publish. In the meantime, NewsDiffs fills the role. We have lots of data, ready to be mined for useful information.Eric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 29 / 30
  60. 60. Outline of Talk 1 Motivation and Creation 2 Case Studies 3 Future 4 Q&AEric Price, Margaret Sullivan (MIT, NYT) NewsDiffs: Version Controlling the News 2013-03-11 30 / 30

×