What publishers need to know about digitization

What publishers need to know about digitization Liza Daly Consultant, Threepress Consulting Inc. http://threepress.org/ Thursday, November 13, 2008

Introduction Liza Daly liza@threepress.org Software engineer and consultant specializing in web-based publishing applications Digitization projects for Ford Foundation, Arnold Arboretum, Rosen Publishing and SAGE Publications Online reference products for Oxford University Press and Columbia University Press Current: ebook applications and consulting Thursday, November 13, 2008

Introduction What I’ll cover 1. Digitization 101: from scanning to OCR to XML 2. Smart vendor selection 3. A gentle introduction to XML 4. I’ve got digital content: now what? ? Thursday, November 13, 2008

What we talk about when we talk about digitization Turning printed content... text ...or microﬁlm archives ...or documents in legacy systems ...into modern digital forms. (sometimes starting from print is easier) <text> Thursday, November 13, 2008

Digitization 101 Assume that we’re starting from a print archive. (If you’re starting from a digital ﬁle, congratulations, your costs just went down -- but not to zero!) Thursday, November 13, 2008

Scan From paper to digital images... Thursday, November 13, 2008

OCR ...to digital text... Thursday, November 13, 2008

XML ...to reusable markup. Thursday, November 13, 2008

Digitization 101 Scanning http://www.ﬂickr.com/photos/heather-dietz/448629362/ Thursday, November 13, 2008

Digitization 101 Scanning Scan http://www.ﬂickr.com/photos/heather-dietz/448629362/ Thursday, November 13, 2008

Digitization 101 Scanning methods Destructive scanning Pages are cut out of the binding and machine-fed into the scanner in batch. (Imagine a huge ofﬁce copier.) Scanned copies are normally destroyed. Thursday, November 13, 2008

Digitization 101 Scanning methods Non-destructive scanning Pages kept in their original binding Manual page-turning Originals are returned to the source Primarily for rare or historical works Thursday, November 13, 2008

Digitization 101 Scanning methods High-volume, non-destructive automated scanning also exists. Thursday, November 13, 2008

Digitization 101 OCR Optical Character Recognition OCR software “guesses” the letters that appear in an image. A dictionary is used to help correct errors. Common errors include wordsruntogether or speling mistakes. Thursday, November 13, 2008

Digitization 101 OCR OCR quality is sensitive to a number of factors. Is the document in good condition with clear type? Is the layout simple or complex? Is a custom dictionary required for proper names or obscure terms? Thursday, November 13, 2008

This is easy. Thursday, November 13, 2008

This is hard. Thursday, November 13, 2008

http://timesmachine.nytimes.com/ Thursday, November 13, 2008

Digitization 101 OCR Better OCR Worse OCR Multicolumn, Layout Simple text sidebars Vocabulary Common Specialized Damaged, dirty or Source quality Clean and legible partial Thursday, November 13, 2008

Digitization 101 OCR Limitations and cautions: Documents with specialized jargon, such as medical journals or archaic texts, will require custom dictionaries. Tables and equations aren’t suitable for OCR. A human check is always advisable. Thursday, November 13, 2008

If the goal of digitization is to make content ﬁndable on the web, the text needs to be correct. Thursday, November 13, 2008

SCAN the documents to convert to digital ﬁles Apply OCR to the scans to get computer-ready text Convert the text into XML X Thursday, November 13, 2008

Digitization 101 XML Not all digitization projects end with XML. Why? Thursday, November 13, 2008

Characters-per-page versus digitization cost/time 1,000 1,500 2,000 3,000+ XML Human-checked OCR Machine OCR Thursday, November 13, 2008

Vendor selection and costs Thursday, November 13, 2008

Consider: But also: Quantity of material Project management Quality of the originals Shipping Layout complexity Heterogeneous content Vocabulary Front/back matter & indexes Thursday, November 13, 2008

Vendor tips Send samples before considering any estimate ...and have the output evaluated. Compare not just cost-per-page but estimated time. Feel comfortable with their project management. Check references! Thursday, November 13, 2008

Should you partner? Thursday, November 13, 2008

? Thursday, November 13, 2008

? ? Thursday, November 13, 2008

It’s too early to say whether Google Books is right for all publishers. But you’re certainly giving up: 1. Control 2. Revenue share 3. Ownership Thursday, November 13, 2008

Creative partnerships Consider whether some of your backlist is public domain or can be released under a Creative Commons license. Thursday, November 13, 2008

XML 101 Thursday, November 13, 2008

XML 101 What’s XML? XML is just plain text, with markers to tell a computer what the text means and how it should be laid out. Thursday, November 13, 2008

XML 101 What’s XML? Text with “markup” is an old idea. This is a paragraph.¶ This is another paragraph. Thursday, November 13, 2008

XML 101 What’s XML? XML just changes the symbols around. This is a paragraph. This is another paragraph. Thursday, November 13, 2008

XML 101 What’s XML good for? 1. Everybody speaks it. 2. Once you have one kind of XML, it’s easy to turn it into another kind. Thursday, November 13, 2008

When you decide to digitize to XML, you’ll need to pick what kind of XML you want. Thursday, November 13, 2008

Kinds of XML Thursday, November 13, 2008

Kinds of XML DTD Thursday, November 13, 2008

Kinds of XML Language DTD Thursday, November 13, 2008

Kinds of XML Language DTD Format Thursday, November 13, 2008

Kinds of XML Language DTD Schema Format Thursday, November 13, 2008

Kinds of XML Language DTD Schema Format XSD Thursday, November 13, 2008

XML 101 Schema vocabulary The schema deﬁnes the list of <tags> that appear in a document, and what they mean. A paragraph ¶ in one schema might be , but in another it might be <para>. Thursday, November 13, 2008

METS/ DocBook ALTO ePub PRISM DAISY TEI Thursday, November 13, 2008

METS/ DocBook ALTO ePub XML PRISM DAISY TEI Thursday, November 13, 2008

XML 101 Choosing a schema Books DocBook, DAISY, ePub, TEI Magazines/ Newspapers METS/ALTO, PRISM Scholarly TEI, MathML Thursday, November 13, 2008

XML 101 DIY schemas Creating your own schema should be a last resort. Expensive to build and maintain. High training and hiring costs. Reduced opportunities for interoperability. Regulatory compliance. Thursday, November 13, 2008

Complex schemas cost more... $$$ $ Low High ...but also provide more opportunity for product development. Thursday, November 13, 2008

Now what? Thursday, November 13, 2008

Monetizing XML conversion XML Thursday, November 13, 2008

Monetizing XML conversion XML web Thursday, November 13, 2008

XML web Thursday, November 13, 2008

UGC web Thursday, November 13, 2008

Remixing content XML allows content to be distributed, altered, and recontextualized in unexpected ways. http://ﬂickr.com/photos/thomashawk/2492298772/ Thursday, November 13, 2008

Small Beer Press Thursday, November 13, 2008

Questions? Liza Daly Threepress Consulting Inc. +01 617 301 0552 liza@threepress.org Thursday, November 13, 2008

http://par-dela.blogspot.com	378
http://toc.oreilly.com	194
http://blogs.oreilly.com	150
http://blog.threepress.org	115
http://par-dela.blogspot.fr	27
http://caosordenado.com	21
http://www.slideshare.net	18
http://eduflabs.com	16
http://didactice.wordpress.com	10
http://radar.oreilly.com	9
http://translate.googleusercontent.com	9
http://pintini.blogspirit.com	3
http://didactice.me	3
http://par-dela.blogspot.ca	2
http://feeds.feedburner.com	1
http://par-dela.blogspot.in	1
http://indexmb.com	1
http://mobilnetexty.pl	1
http://pintiniblog.wordpress.com	1
http://noticiasediciondigital.malaletra.com	1
http://par-dela.blogspot.it	1

What publishers need to know about digitization

by lizadaly on Nov 13, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

21 Embeds 962

Statistics

What publishers need to know about digitization — Presentation Transcript