Nuance-ACEDS May 21 OCR Webcast

© 2002-2013 Nuance Communications, Inc. All rights reserved. Page 1
How to Keep OCR
Errors from
Spoiling Your
eDiscovery Party
ACEDS Webinar, May 21st 2014

ACEDS Membership Benefits
Training, Resources and Networking for the
E-Discovery Community
Join Today! aceds.org/join
or Call ACEDS Member Services 786-517-2701
Exclusive News and Analysis
Weekly Web Seminars
Podcasts
On-Demand Training
Networking
Resources
Jobs Board & Career Center
bits + bytes Newsletter
CEDS Certification
And Much More!
“ACEDS provides an excellent, much needed forum… to train, network and stay
current on critical information.”
Kimarie Stratos, General Counsel, Memorial Health Systems, Ft. Lauderdale

Speaker Introduction
Greg Gies
Director, Product Marketing, Imaging
Nuance Communications
Leads go-to-market planning for
Nuance’s print, capture & PDF
solutions.

Agenda
– What OCR is.
– What are OCR errors and what causes them.
– Why eDiscovery professionals should care.
– How common these problems are.
– What can be done to prevent OCR errors.
– How to correct OCR errors when they happen.

What is OCR?
Wikipedia: “Optical character recognition, usually
abbreviated to OCR, is the mechanical or electronic
conversion of scanned or photographed images of
typewritten or printed text into machine encoded /
computer-readable text.”
2009 Computerworld Article: “Optical character
recognition (OCR) is the translation of optically scanned
bitmaps of printed or written text characters into character
codes.”

I say, “OCR is the digital transcription of
bitmaps containing machine-printed text
into encoded text characters, using a
coding scheme such asASCII, which
among other capabilities enables indexing
software to decipher textual elements
contained within bitmaps.
Encoded text is ‘searchable’text.

What are OCR errors and what causes
them.

Types of OCR errors
Transcription errors: Result: misspelled words.
Impact: Unsearchable. Proportional fonts are
especially problematic.
Example, “learning” becomes “leaming”
Formatting errors. Result: poor legibility.
Impact: Unsearchable.
Example, “learning” becomes “l e a r n i n g”
Deleted metadata: Result: data loss.
Impact. Metadata is unsearchable.

Some image defects and causes
DEFECT CAUSE
Faulty
printing
equipment
Toner specks
Vertical lines
Toner smear
Gray background
Page skew
Light/dark print
Defective toner cartridge
Worn rollers
Wrong settings
Worn pickup roller
Low toner or ink cartridge
Clogged nozzles
Paper or
form
elements
Halftones
Vertical/horizontal lines
Noise
Colored paper
Carbon copies
Shaded and lined forms
Low/high contrast background
Faulty
scanning
equipment
Specks
Page skew
Light/dark image
Dirty platen
Feeder misadjusted
Worn pickup roller
Low resolution
Misfeed

Examples of image defects
Halftone
Specks
Color Background
Fuzzy edges
Gridlines

Cause & effect
SKEWED IMAGE
TONER SPECKS
DARK TEXT

OCR results
•..... ; .,
1HIt#~!':tl'Ol<fLoi;l;i~: •••• 'do ••••• ~""r.bf .. . ,200., by- ••• ~ •• ODNEY
D~"'!Ob$Y~s.IN(; ..• ~~"""""" Iaws "'''''.S_",c.rm",,;, ~,-"'"'_U "s.,.",) •••
JOE STANDUi' ~Irnown •• """,,"). _ and SclIa _<01"'"""y
'l>eknoWhllereinasutlJ.e paiti~S",' . . . ..... ..
. "

Formatting errors
Original Image OCR conversion to .docx

Why should eDiscovery professionals
care about OCR errors?

PDF Files Created by ScannersAren’t
Necessarily Searchable
1. Unlike PDF Files that were “born digital”, a PDF file from
a scanner is an image of a paper document.
2. While the text in an image may appear similar or the
same as text in a “born digital” PDF, it’s invisible as far
as search algorithms are concerned.
3. Images aren’t searchable until processed with OCR.
4. “Searchable PDF” contains OCR output in its metadata.

Digitized Versus Native Documents
Electronic
• Originated online
• Many native file formats
• Encoded text – “machine-
readable”
• eDiscovery software designed
for these documents
Digitized
• Scans of paper originals
• TIFF & PDF common formats
• Text is a bitmap; not encoded
• OCR makes scans “machine-
readable”
Why special handling is required

All Searchable PDFsAren’t Equally
Findable

All Searchable PDFsAren’t Equally
Findable
– OCR Word Accuracy Matters
– Character vs. word accuracy
– 1character error = 1 word error
– 1 char / 10,000 = .0001
– 1 word / 1,000 = .001
– Seemingly small differences in
OCR error rate lead to very large
differences in errors
– 98.25% vs. 96.05%
– 948 pages
– ~ 2% delta
– Over 12,000 more word errors

PDF Isn’t Just PDFAnymore
Improper handling may lead to data loss
Electronic “sticky
notes” & stamps
obscure text
behind
PDF contains
digitized pages
or elements
OCR software may
flatten the image,
i.e., convert PDF to
TIF, then OCR
PDF contains
native pages or
elements

Four Important Ways PDF is Different
1. PDF files can be assembled from multiple files.

2. PDF elements can be rearranged.
User copied snip of
scanned document.

3. PDF files can have multiple layers.

4. PDF files can be “born digitally” or created by a scanner.

Why These Differences Matter
When these properties converge with scanned
text eDiscovery pitfalls arise that can lead to
inadvertent data loss due to processing errors.

Single PDF Page Can Contain Both
“Born Digital” and Scanned Content
This PDF
contains 2
scanned pages.

Notes, Text Boxes, Callouts, Stamps,
Etc…Can Be Overlaid On Images

How Data Gets Inadvertently Destroyed
OCR may ‘flatten’ objects overlaid on text, making the text
underneath unreadable and metadata unsearchable

Why Is Data DestructionA Problem?
“Spoliation is the destruction or significant alteration of
evidence, or the failure to preserve property for another's
use as evidence in pending or reasonably foreseeable
litigation.”
Source: http://www.gwblawfirm.com/ap-spoliation-a-trap-for-the-unwary.php

Penalties for Spoliation
$2,750,000
United States v. Philip Morris USA, Inc.

What If The Data Was Destroyed
Unintentionally? I’m Okay Right?
“The intent to alter or destroy electronic data is
not required for spoliation to occur.”

Other Reasons Data Loss is Bad
– Destruction of evidence – affect case outcome.
– Time wasted recreating lost data – reduced profitability.
– Client relations – loss of credibility and future business.
Spoliation isn’t the only concern

How common are OCR errors?

Typical OCR Word Accuracy Rates

Firms With Initiatives To Convert Paper
DocumentsAnd Process To Digital
0%
20%
40%
60%
80%
100%
Total
PercentageofRespondents
Don’t know
No
Yes

What Percentage Of The Knowledge
Workers Have Access To Scanners?
72.8%
66.7%
75.2%
64.5%
78.6%
80.9%
0%
20%
40%
60%
80%
100%
Total Education Financial Healthcare Insurance Legal
Workers

The Use Of Scanning Within
Organizations
0%
20%
40%
60%
80%
100%
Total
Decreasing
Stay the same
Increasing

Why scanning is increasing
0%
20%
40%
60%
80%
100%
Total
More people have
been given access
to scanning
Mix of more
documents being
scanned and more
people gaining
access to scanning
Individuals are
scanning more
documents
N = 158
Respondents who have seen an increase in scanning

What can be done to prevent OCR
errors from occurring

Image enhancement filters out defects
Halftone
Removal
Despeckle
Color
Background
Removal
Smooth Characters
Remove Gridlines

Selectively Processing PDF Files Isn’t a
Viable Strategy
– Client delivers a disk with millions of
files, how do you know which PDF
are “compound” that require special
handling?
– It’s kind of like looking for needles in
a haystack.

SegregateAll PDF Files Before
Conversion
– After collecting documents,…
– …but before processing, review and analysis
– Identify all PDF documents
– Move them to a separate directory to be run through OCR

Pre-process all PDF files with OCR
– Text images are potentially hidden within files
– Newer OCR tools will make image text searchable,
without disturbing the rest of the data within the document
– Ensures all text within every PDF document can be
searched by eDiscovery system
– Should also convert to PDF/A at the same time so files
are ready for court filing

How to correct OCR errors when they
happen

Post processing error-correction
options
1. Proofreading – too time-intensive
2. Run an automated spell check
– Effective at identifying and correcting spelling errors
– Doesn’t solve contextual errors
 Example, “How is you day?”
 Spelled correctly but clearly is wrong
 If searching for “your” this instance won’t be found
 No commercial solutions today solve this problem
 Helps to understand this potential problem
 Can either manually review and correct or adjust search strategy to
minimize the impact, e.g., fuzzy search techniques

Final Thought – Why You Should Care
General business /
transaction documents
Purchase orders, invoices, contracts, employee identification documents…
Healthcare organizations
/ patient medical records
Physician’s notes, discharge summaries, test results, post operative reports, etc…
Personal identification
documents
Driver’s licenses, social security cards, professional certificates
Public institutions /
Police records
Incident / accident reports, police logs, court records…
Insurance & banking Claims documents, medical records, financial records…
Schools Inoculation records, transcripts, applications…
Important enough to print, then to scan, it is important!

Q&A

Next Steps
• A recording of today’s Webinar will be available shortly for you to review
at your leisure
• Contact Nuance with any questions you may have:
781-565-5000 or imaging@nuance.com
• For more information visit: http://www.nuance.com/for-business/by-
industry/legal/legal-solution
• Try our 30-Day free trial of Power PDF at http://www.powerpdf.com

Thank you

Nuance-ACEDS May 21 OCR Webcast

Recommended

Recommended

More Related Content

Similar to Nuance-ACEDS May 21 OCR Webcast

Similar to Nuance-ACEDS May 21 OCR Webcast (20)

More from Robbie Hilson

More from Robbie Hilson (7)

Recently uploaded

Recently uploaded (20)

Nuance-ACEDS May 21 OCR Webcast