Diy Digitization of Community Newspapers

Art Rhyno, University of Windsor/
OurDigitalWorld
OCUL Digital Curation Summit - Oct. 27, 2016OLA Super Conference - Jan. 31, 2018

It has been estimated
that newspaper
digitization can cost
an organization $3 US
per page *
* “Amateur Beats Gov't at Digitizing Newspapers: Tom Tryniski’s
Weird, Wonderful Website” -
http://reason.com/archives/2013/03/05/amateur-beats-gov-at-
digitizing-newspape/1
- ink.ourdigitalworld.org

BOOKS
NEWSPAPERS
The Essex Free Press (weekly) -
70,335 pages for 1895 - 2011
● Microfilm: 41,073
● Microfiche: 22,982
● PDF: 6,280

Small Sources of
Big Data
Title Coverage Pages
The Amherstburg Courier 1849-1850 183
The Amherstburg Echo 1874-1982 46,024
The Border Cities Star (Windsor) 1918-1935 144,965
Canadian Emigrant And Western District Advertiser (Windsor) 1831-1836 365
Le Canadien (Windsor) 1891 3
The Comber Herald 1892-1917 5,768
Commonwealth (Windsor) 1857 8
Le Courrier and Le Courrier De L'Ouest (Windsor) 1883 - 1909 34
Courrier D'Essex 1884-1885 11
La Defense (Windsor) 1918 5
The Essex Free Press 1895-2011 70,335
The Essex Record 1871-1882 1,987
The Harrow News 1950-2011 31,495
L'Independent Du Canada (Windsor) 1892 4
The Kingsville Reporter 1893 - 2001 78,237
LaSalle Silhouette 1990-2011 46,982
The Leamington Post 1907 - 1995 104,165
The Provincial Freeman 1853 - 1857 278
The Tecumseh Maple Leaf 1931-1958 10,685
The Tecumseh Tribune 1959-2010 11,127
True Royalist (Windsor) 1860 - 1861 25
The Voice of the Fugitive 1851 - 1852 131
The Walkerville Mercury 1890 - 1891 206
The Walkerville News 1934 - 1936 477
The Walkerville Times 1999-2016 1,613
The Western Herald (Windsor) 1838 - 1842 755
The Windsor Evening Record 1893 -1918 71,789
The Windsor Herald 1855 - 1856 420
628,077

95. The Weekly Post (Lindsay) 1900-1907 4,780
96. Weekly Sun (Toronto) 1895-1896 5,472
97. The Welland Canal 1835 12
98. Western Dispatches (Michigan) 1817-1854 4,823
99. The Western Herald (Windsor) 1838 - 1842 755
100. Western planet (Chatham) 1853-1856 1,240
101. The Western Sentinel (Chatham) 1847 320
102. The Whitby Free Press 1971-1996 34,422
103. The Whitby Gazette and Chronicle 1912-1941 2,795
104. Wilmette Life 1923-1966 45,831
105. The Windsor Evening Record 1893 -1918 71,789
106. The Windsor Herald 1855 - 1856 420
107. The York Commonwealth 1858-1963 997
108. The York Herald 1859-1890 946
1,716,934
No. Title Coverage Pages
1. The Acton Free Press 1875-1984 33,643
2. The Acton Tanner 1992-2001 3,154
...
Ready to load...

The Opportunities
& Challenges of
Microform

Product Cost Lease Description
ScanPro 3000
Microfilm Scanner
$10,895 US for
hardware/ $1,200 for
base software,
$3,500 AutoScan
software (for
scanning of evenly
spaced microfilm)
$452.60 US per
month (3 year)
$306.33 US per
month (5 year)
Designed for both
staff-side and public
scanning. Limited
options for batch
scanning but frame
by frame scanning is
supported.
Mekel Mach 5
Microfilm Scanner
$49, 900 to 69,900
US/ service contract
varies
15% of upfront cost,
minimum 3 year
contract
Designed for batch
processing, entry
model.
Mekel Mach 7
Microfiche Scanner
$69, 900 to 89,900
US/ service contract
varies
15% of upfront cost,
minimum 3 year
contract
Microfiche scanning
is typically more
costly than with
microfilm and some
services only offer
microfilm
conversion.

Goal - CREATE A $1000 MICROFILM SCANNER!
● exclude workstation from budget
● use OSS tools & applications
● use software to cover gaps in
deficiencies in hardware
● scan a reel a day unattended
● how hard could it be?

● Dealing with torque
● Configuring a stable camera
arrangement
● Supplying appropriate lighting
● Finding flexible options for
merging
Top 4 Challenges to Automated
Scanning of Microfilm

Torque
“a twisting force to cause rotation”
~$60 ~$50 ~$100

Cameras - the trickiest bit...
The problem with DSLR cameras

Cameras - the trickiest bit (cont.)
When a low-cost macro lens is not enough...

Mirrorless cameras - candidate 1:
Canon M3 (~$500)

Mirrorless cameras - candidate 2: Canon M5 (~$1000)
M6 now available (~$700)

EF-M 28mm f/3.5
Macro IS STM lens
(~$350)

High-end
Scanning
Service
Macro
Photography

Lighting - the easiest bit
● Designers Edge
Portable Work Light
($75)
● 779 lumens (on the
low end, watch out
for blinding levels)

Merging Software
● Image Composite Editor (windows) -
https://www.microsoft.com/en-
us/download/details.aspx?id=52459
● Panorama stitching tools (linux) -
https://wiki.panotools.org/Panorama_scripting_in_a_nutshell
● AutoStitch (windows) -
http://matthewalunbrown.com/autostitch/autostitch.html

Enabling Software
● Gphoto2 - http://gphoto.org
gphoto2 --capture-preview
● OpenCV - https://opencv.org
cv2.drawContours(img, [approx],
-1, (0, 255, 0), 50)
● ImageMagick - https://www.imagemagick.org
convert -black-threshold 75% 00001.jpg 00001_proc.jpg

Goal - CREATE A $1000 MICROFILM SCANNER!
● exclude workstation from budget
● use OSS tools & applications
● use software to cover gaps in
deficiencies in hardware
● scan a reel a day unattended
● how hard could it be?
$2000
~

Gotchas
● Camera settings -
understanding aperture, using
landscape lighting
● Microfilm can be remarkably
inconsistent
● Dealing with endless streams
of data

And some other directions that
might have promise...
● CHDK -
http://chdk.wikia.co
m/wiki/CHDK
● Are there not
other camera
manufacturers?

● Olena for page segmentation -
https://www.lrde.epita.fr/wiki/Olena/
WebHome
In addition to scanning...
● Abbyy Finereader -
https://www.abbyy.com
/en-ca/finereader/
● Tesseract OCR -
https://github.com/tess
eract-ocr/tesseract
tesseract --oem 2
● Google Cloud Vision
API OCR -
https://cloud.google.co
m/vision/docs/ocr

Thank you!
And please don't forget to visit the ODW booth - #707

Diy Digitization of Community Newspapers

Recommended

Recommended

More Related Content

More from OurDigitalWorld

More from OurDigitalWorld (17)

Recently uploaded

Recently uploaded (20)

Diy Digitization of Community Newspapers