• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
IMPACT Final Conference - Gregory Crane
 

IMPACT Final Conference - Gregory Crane

on

  • 2,377 views

 

Statistics

Views

Total Views
2,377
Views on SlideShare
1,964
Embed Views
413

Actions

Likes
0
Downloads
3
Comments
0

2 Embeds 413

http://impactocr.wordpress.com 410
http://www.digitisation.eu 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Checked Shakespeare Quarterly 61.1 Spring 2011 – of 100 randomly selected footnotes with secondary source citations, 99 cited English. 1 cited french. A handful cited Greek and Latin sources but in each case cited English translations. There were 4 articles and a review essay
  • So “a” solution that we’re presenting here is the idea of transferring markup from a richly annotated source text to a plain-text translation (that can be nothing more than perhaps badly OCR’d words). So at a high level, this is done in two steps: an alignment step and a projection step. First we align the source document with the target document in a cascading process and when we have this alignment completed, we then project the XML tags across documents in a way that exploits the similarity in the linguistic structure between that text pair.
  • 1. ocr outputs are aligned by a multiple sequence alignment dynamic algorithm (similar to BLAST, Clustal, etc., used in bioinformatics for DNA alignment) -- 2. the bayesian classifier determines, character by character, which character is more reliable, taking into account the probability that a particular engine provides that good character, when the other engines provides their bad characters (ex.: e1=R e2=P e3=B <--house [h-o-u-s-e are provided, in the correct position, at least by one engine:h in e3, o in everyone, u in e1, etc.)]]

IMPACT Final Conference - Gregory Crane IMPACT Final Conference - Gregory Crane Presentation Transcript

  • OCR and the Transformation of the Humanities Gregory Crane and David Bamman Tufts University Bruce Robertson Mount Allison University John Darlington and Brian Fuchs Imperial College London
  • Three basic changes
  • Three basic changes
    • Transformation of scale of questions
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
    • Student researchers and citizen scholars
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
    • Student researchers and citizen scholars
        • Not enough professors and library professionals
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
    • Student researchers and citizen scholars
        • Not enough professors and library professionals
    • Globalization of cultural heritage
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
    • Student researchers and citizen scholars
        • Not enough professors and library professionals
    • Globalization of cultural heritage
      • Not enough expertise in Europe + North America
  • Towards Dynamic Variorum Editions Gregory Crane and David Bamman Tufts University Bruce Robertson Mount Allison University John Darlington and Brian Fuchs Imperial College London
  • Thanks to …
    • Digging into Data Phase 1
    • National Endowment for the Humanities
    • JISC (UK)
    • SSHRC (Canada)
    • National Science Foundation
    • Mellon Foundation
    • Google Digital Humanities
    • Cantus Foundation
    • German Research Foundation
  • The Dynamic Variorum as grand challenge
  • The Dynamic Variorum as grand challenge
    • How do you build self-organizing collections?
  • What is a variorum?
    • Short for cum notis variorum , “with notes of different people”
  • New Variorum Shakespeare Series
  • New Variorum Shakespeare Series
  • New Variorum Shakespeare Series “ New” = 140 years old
  • New Variorum Shakespeare Series “ New” = 140 years old “ New” vs. 1821 Shakespeare Variorum
  • Heinsius’ Claudian
  • Heinsius’ Claudian
  • Heinsius’ Claudian
  • NVS 2011
  • NVS 2011
  • What was in the 1873 NVS Macbeth?
  • What was in the 1873 NVS Macbeth?
    • Index
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
    • Bibliographies
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
    • Bibliographies
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
    • Bibliographies
    • Running Text
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
    • Bibliographies
    • Running Text
    • Multiple Versions
  • What was in the 1873 NVS Macbeth?
    • Index
    • [Table of contents]
    • Sources
    • Adaptations
    • General Topics
    • Bibliographies
    • Running Text
    • Multiple Versions
    • Annotations
  • Brown’s Intermedia c. 1990
  • The problem…
    • Not feasible to summarize scholarship on any major canonical author by manual means
    • An issue in 1665 and in 1905 but much worse now…
    • How do we generate a Variorum edition from the very large collections that make this such a challenge? How do we make scale an advantage?
  • Shakespeare as an easy case…
  • Shakespeare as an easy case…
  • Shakespeare as an easy case… c. 500 years of English ….
  • Greco-Roman World
  • Greco-Roman World From Rabat to Kandahar …
  • c. 100 CE papyrus from Euclid (c. 300 BCE) http://www.math.ubc.ca/~cass/Euclid/papyrus/papyrus.html
  • 800-1000 CE: Greek into Arabic Hunayn Ibn Ishaq (809–873), Arabic version of the Prognosticon from the Hippocratic Corpus http://www.nlm.nih.gov/exhibition/odysseyofknowledge/
  • c. 1200-1300: Arabic into Latin Medieval Translation of the Prognosticon from Arabic into Latin
  • Return of Greek sources c. 1500 This first edition of Dioscorides' Greek text, printed in Venice in 1499 by Aldo Manuzio (ca. 1447–1515)
  • Status as of October 2011
    • What do you do with a billion words?
      • 2000 years of Latin
    • How do you integrate data across languages
      • Projecting markup over noisy data
    • How do you trace ideas?
      • Detecting changes within and across languages
    • How do you get the data you need?
      • Customizing OCR for a pre-modern language
    • How do you scale up your services?
      • From workflows to Cloud-based design
  • Disciplines and Speakers
    • David Bamman, Tufts University
      • Computational Linguistics
    • Bruce Robertson, Mount Allison University
      • Digital Classics
    • Brian Fuchs, Imperial College London
      • Software Engineering
  • 1. Computational Linguistics David Bamman Tufts University (Carnegie Mellon University) United States
  • Overview: Publications
      • Bamman, David and Gregory Crane (2011), “Measuring Historical Word Sense Variation,” Proceedings of the 11 th ACM/IEEE Joint Conference on Digital Libraries (JCDL 2011). Nominee, Best Paper Award.
      • Bamman, David, Alison Babeu, and Gregory Crane (2010), "Transferring Structural Markup Across Translations Using Multilingual Alignment and Projection," in: Proceedings of the 10th ACM/IEEE Joint Conference on Digital Libraries (JCDL 2010). Winner, Best Paper Award.
      • Bamman, David and David Smith (forthcoming), “Extracting Two Thousand Years of Latin from a Million Book Library”, Journal of Computing and Cultural Heritage.
  •  
  • 2000+ Years of Latin
  • Goal: Tracking Language Change
    • Lexical change (new vocabulary, shifts in word sense)
    • Syntactic change (including the influence of the author’s L1 on the Latin syntax)
    • Topical change (the rise of new genres)
    • Identifying the spread of variation across authors.
  • Corpus development
    • Data source
      • 1.2M books from the Internet Archive (snapshot of collection from 2009)
      • 25,886 works catalogued as Latin
    • Metadata problems
      • 1. Language identification (many of these works are not Latin.)
      • 2. Historical date info (dates of publication != dates of composition.)
  • 25,886 works catalogued as Latin in the IA, charted by “date.”
  •  
  •  
  • Language ID
    • Language ID to identify which of these works actually have Latin as a major language.
      • Trained a language classifier (alias-i Lingpipe) on:
        • 24 editions of Wikipedia
        • Perseus classical corpus
        • Known badly-OCR’d Greek in the IA.
    • Results
      • 10,263 of 25,886 books catalogued as Latin are not recognizably so (mostly Greek)
      • 6,790 books not catalogued as Latin in the 1.2M collection are in fact so (98% precision).
      • Net: 22,413 Latin books containing 2.97 billion words .
  • Composition dating
    • With undergraduate students in Classics, established dates of composition for each Latin text. So far, considered 10,398 of them:
      • 7,055 dated
      • 3,343 excluded as not representative of language use – e.g., reference works (dictionaries, catalogues, lists of manuscripts)
    • From these 7,055 works, we extract just the Latin to create a dated historical corpus of 389 million words.
  • 25,886 works catalogued as Latin in the IA, charted by “date.”
  • 7,055 Latin works in the IA, charted by date of composition.
  • “ America” (1066)
  • “ de” (2,955,462)
  • “ oratio”
  • “ lead” vs. “iron”
  • Polysemy Words have many senses. Lead Iron (verb) cause to go (verb) to smooth w. an iron be in command (noun) element Fe (noun) position of advantage tool with flat steel base used to smooth clothes chief part in play golf club element Pb graphite in pencil Oratio (noun) Speech Prayer
  • Measuring sense variation
    • Method: Train broad-coverage word sense disambiguation using aligned parallel texts
    • English/French (Diab and Resnik 02), English/Chinese (Chan and Ng 05, Ng et al. 03), English/Portuguese (Specia et al. 05), English/Vietnamese (Dinh 02).
    • Parallel text alignment
    • Identify translations (130 English translations manually identified by students from a representative range of dates)
    • Word align Latin text <-> English text (ca. 1.3M words)
    • Induce a sense inventory from the alignment
    • Word sense disambiguation
    • Train a WSD classifier on noisily aligned texts
    • Automatically classify remaining 387M words
    • Track lexical change
  • WSD via parallel texts
    • SMT based on Brown et al (1990)
    • Different senses for a word in one language are translated by different words in another.
    • “ Bank” (English)
      • financial institution = French “banque”
      • side of a river = French “rive” (e.g., la rive gauche )
  • (Dynamic Lexicon)
  • (Bootstrapping multilingual digital libraries) +
    • Projecting XML markup across editions and translations (Bamman and Crane 2010)
    • Alignment of the source document with the target document in a cascading process: document -> sentence -> word
    • Projection of XML tags in the source document to the target document in way that exploits the linguistic similarity of the text pair.
  • 2. Parallel text alignment
    • Sentence level : Moore’s Bilingual Sentence Aligner (Moore 2002)
      • aligns sentences that are 1-1 translations of each other w/ high precision (98.5% on a corpus of 10K English-Hindi sentences)
    • Word level : MGIZA++ (Gao and Vogel 2008)
      • parallel version of: GIZA++ (Och and Ney 2003) - implementation of IBM Models 1-5.
  • 3. Sense induction
  • 4. WSD Training Source word oratione (oratio) Sense label prayer Training context ad spem pertinent, quae in … dominica continentur
  • 5. WSD Classification
    • For all words without an aligned translation, use the surrounding context to determine the most likely sense.
  • 5a. WSD static evaluation
    • Created held-out test set of 105 instances of 5 Latin nouns with known shifts in meaning sampled uniformly from 21 centuries. Evaluated 7 different WSD classifiers + simple baseline of most frequent sense overall (MFS).
    System villa pastor miles scientia oratio Average 5-gram LM 54.8% 69.2% 90.2% 73.7% 61.4% 69.9% 6-gram LM 58.3% 61.5% 91.2% 65.8% 63.8% 68.1% Bayes 63.5% 62.3% 92.6% 70.2% 48.0% 67.3% Token Unigram LM 63.5% 62.4% 92.6% 70.2% 48.0% 67.3% Token Bigram LM 64.3% 62.4% 92.6% 70.2% 48.8% 67.7% TF/IDF 64.3% 60.7% 82.8% 70.2% 49.6% 65.5% KNN 64.3% 73.5% 84.4% 63.2% 40.1% 65.1% MFS Baseline 60.9% 66.7% 92.6% 79.0% 60.6% 72.0%
  • 5b. WSD time series evaluation
  • 5b. WSD time series evaluation
  • 5b. WSD time series evaluation
    • Evaluated via mean square error between gold standard time series and automatically classified one.
    System villa pastor miles scientia oratio Average 5-gram LM .056 .034 .052 .044 .137 .065 6-gram LM .053 .053 .052 .022 .022 .040 Bayes .047 .060 .055 .040 .228 .086 Token Unigram LM .047 .060 .055 .044 .230 .086 Token Bigram LM .047 .060 .055 .044 .230 .087 TF/IDF .037 .050 .049 .040 .189 .073 KNN .101 .028 .054 .039 .248 .094 MFS Baseline .228 .170 .014 .091 .338 .178
  • “ oratio”
  • 6. Tracking lexical change: “oratio”
  • Acknowledgments
    • This work was supported by grants from:
      • The Digging into Data Challenge (&quot;Towards Dynamic Variorum Editions”)
      • The National Science Foundation (IIS-910884, &quot;Mining a Million Scanned Books: Linguistic and Structure Analysis, Fast Expanded Search, and Improved OCR”)
      • The National Endowment for the Humanities (PR-50013-08, &quot;The Dynamic Lexicon: Cyberinfrastructure and the Automated Analysis of Historical Languages”)
    • Thanks are also due to research assistants Alison Darling, Elise Goodman-Tuchmayer, Daniel Libatique, Lee Marmor, John Owen and Erin Shanahan.
  • 2. Digital Classics Bruce Robertson Mount Allison University Canada
  • Digitizing and Viewing Difficult Texts: Lessons From Ancient Greek   19th century provides a vast array of editions of Greek text, many still very useful          - Yet they could not be accessed digitally   What tools and workflows might help us digitize diverse texts such as these? What applications can we create to make the resulting OCR data useful to researchers and students?
  • Diversity of 19th Century Fonts and Layout
  •  
  • Character Classification and the Modern Undergraduate
    • Performing optical character recognition requires a great deal of 'training'
    • This is perfectly suited to the undergraduate researcher
    • Ph.D. student asks: &quot;why isn't this part of the beginning Greek curriculum?&quot;
    •     It introduces students to the beauty and heritage of the typography of their subject
    •     It immediately engages them in a vital research project
    • (True of all languages where learning a new character set is a preliminary skill)
  • Results
  •  
  • http://www.youtube.com/watch?v=OIjaq7ds2J8
  •  
  • Lessons Learned
    • Undergraduates provide excellent middle-tier academic labour
    •  
    • Shared dictionary data will be fundamental to a cloud-based approach
    • Include as many languages as possible from the beginning
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
  • Future Work
    • Continue to improve Greek OCR engine based on 'Gamera'
    •  
    • Integrate visualization tools that aid students of the language
    •  
    • Implement many dictionaries: English, French, Latin, etc.
    •  
    • Integrate other crowd-sourcing opportunities so interested viewers can:
    •     Verify  or correct dubious OCR results
    •     Identify the grammar or syntax of words
    •       
    •       
    •       
  • 3. Software Engineering John Darlington Brian Fuchs Imperial College London United Kingdom
  • ICL ’ s role in DVE
    • High-throughput infrastructure for
      • OCR for Greek and Latin
      • Text-based Feature extraction
    • E-Science utility computing infrastructure
    • High-level functional interfaces for e-Science.
  • DVE: Context at SCG
    • E-Science Frameworks
      • Grid
      • Cloud
      • Parallel Processing
      • Functional / Declarative approaches
    • Internet services and economics
      • Healthcare
      • Music
      • Mobile Applications
      • Transport
  • OCR parallel challenge
    • The key to OCR at scale is miminising the need for eyeballs.
    • i.e. “ ground-truth ” -- manual checking against the original.
  • Rapid OCR using MapReduce +a Cloud IaaS
  • Infrastructure: State of play
      • 6 node static hadoop testbed
      • 160 node eucalyptus cluster on old opteron chips
      • 20 dual quad-core machines with 16TB storage on fibre.
      • Stack assembled and deployed
      • Initial training sets tested.
  • Throughput Infrastructure
    • Boschetti Aligner
      • OCR post-processing for Greek/Latin developed at PDL 
      • multiple sequence alignment dynamic algorithm ( like BLAST, Clustal, Mr. Bayes)
      • bayesian classifier to select the most probable sequence of characters
      • spell-checking filtered by ocr evidence 
  • Throughput Infrastructure
    • MapReduce
      • = functional Map/Fold.
      • Made famous by Google, but developed by others.
      • Map, then reduce
      • Map: apply a function in parallel to a bunch of key/value pairs.
      • Reduce: apply a function in parallel to each group of similar k/v pair outputs from Map.
  • Throughput Infrastructure
    • MapReduce
      • E.g. count occurrences of words in docs
      • Map( docname, doc.txt))-> ‘ mittitur ’ :1, ‘ cura ’ :1
      • Reduce(word:count)  ‘ mittitur ’ :23, ‘ cura ’ :10,…
  • Throughput Infrastructure
    • MapReduce
      • E.g. count occurrences of words in docs
      • Map:
        • Count the words in 1000 documents (in parallel)
        • map( docname, doc.txt))-> ‘ mittitur ’ :1, ‘ cura ’ :1
      • Reduce
        • Group the output by word, and add up occurrences (in parallel)
        • Reduce(word:count)  ‘ mittitur ’ :23, ‘ cura ’ :10,…
  • Throughput Infrastructure
    • Eucalyptus
      • Open Source Cloud Computing
      • UC, Santa Barbara  Spin-off
      • compatible with Amazon EC2/ S3
      • Supported in Ubuntu as of 10.4.
  • Throughput Infrastructure
    • Hadoop
      • Apache Distributed File System for MapReduce jobs.
      • MapReduce Engine—co-ordinates MapReduce
  • Cluster provisioning
    • Create an image with the whole stack
    • Deploy the image as many times as nodes are required
    • Push required config data to the nodes
    • Turn on
    • Keep storage separate (i.e. don ’ t use hdfs to store data)
  • OCR parallel methods
    • Run parallel jobs on the same scans
    • Score results
    • Use highest score in the next round
  • OCR parallel methods
    • 3 different ocr engines per page
    • x different filters per page
    • x different filters per section of page.
    • = c. 30 runs per scan.
  • OCR vote and error prediction methods Courtesy: Federico Boschetti
  • Alignment voting
    • Map: Run three ocr engines/training sets on each page
      • Gamera
      • Tesseract: training set 1
      • Tesseract: training set 2
    • Reduce:
      • spell check and compare
  • Training set voting
    • Map: Run random pages on all avail. training sets.
    • Reduce: Check against dictionary, and score.
  • Tiling
    • Map: Run several filters over different parts of a page to compensate for local minima = blotches
    • Reduce: score the output and compare.
  • Why Eucalyptus?
    • Scalable
      • Amazon/NGS hybrid possibilities
    • Reuseable
      • Very fast start-up/tear-down.
    • Configurable
      • Quickly configure custom throughput clusters
  • Why MapReduce?
    • “ Shared Nothing ” architecture
    • = suited to “ dumb ” processes like page ocr
    Why Hadoop?
    • Easy to integrate with other FS ’ s, e.g. s3
    • Excellent customisation options
    • Most flexible implementation of MapReduce (cf. GridGain)
  • Why not MapReduce?
    • Requires extensive refactoring.
    • Only a subset of functional possibilities.
    Why not Hadoop?
    • Filesystem is slooooowwww….
    • Resource intensive.
    • Headnode is a bottleneck…
  • Challenges for the future
    • Feature Extraction.e.g.
      • Named Entities
      • Part of Speech tagging
      • Multi-lingual alignment
    • Iteration is hard with distributed systems!
  • Conclusions
  • Three conclusions
    • Increased intellectual range
  • Three conclusions
    • Increased intellectual range
      • Greco-Roman Antiquity is an enabling subject to understand cultural tectonic forces at work today
  •  
  • Plato ’ s Republic and the Guardians The Islamic Republic of Iran and the Guardianship of Islamic Jurists
  • Sometimes Greek philosophy does have an impact.. Plato ’ s Republic and the Guardians The Islamic Republic of Iran and the Guardianship of Islamic Jurists
  • Three conclusions
    • Increased intellectual range
      • Greco-Roman Antiquity is an enabling subject to understand cultural tectonic forces at work today
    • Cultural heritage -> network of cultures
  • Three conclusions
    • Increased intellectual range
      • Greco-Roman Antiquity is an enabling subject to understand cultural tectonic forces at work today
    • Cultural heritage -> network of cultures
      • We share Greco-Roman Cultural Heritage
  • Students of Greek and Latin
  • Students of Greek and Latin
  • Students of Greek and Latin
  • How do we work together?
  • Three conclusions
    • Increased intellectual range
      • Greco-Roman Antiquity is an enabling subject to understand cultural tectonic forces at work today
    • Cultural heritage -> network of cultures
      • We share Greco-Roman Cultural Heritage
    • Decentralized Lab Culture in the Humanities
  • Three conclusions
    • Increased intellectual range
      • Greco-Roman Antiquity is an enabling subject to understand cultural tectonic forces at work today
    • Cultural heritage -> network of cultures
      • We share Greco-Roman Cultural Heritage
    • Decentralized Lab Culture in the Humanities
      • Even/esp. hard subjects need contributions from student researchers and citizen scholars
  • Student Researchers Tufts
  • Student Researchers Tufts Holy Cross
  • Student Researchers Tufts Furman Holy Cross
  • Student Researchers Tufts Furman Holy Cross Houston
  • Student Researchers Tufts Furman Holy Cross Mount Allison Houston
  • Huge Open Collections
    • Provide the net public with physical access to unprecedented bodies of cultural heritage
    • Researchers and automated systems provide initial intellectual access BUT…
    • These alone cannot succeed without student researchers and citizen scholars
  • Three basic changes
    • Transformation of scale of questions
      • Breadth and Depth
    • Student researchers and citizen scholars
        • Not enough professors and library professionals
    • Globalization of cultural heritage
      • Not enough expertise in Europe + North America
  • We can (if we choose) transform our ability to advance the intellectual life of society
  • Thank you!