Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intelligent Content Extraction from PDFs


Published on

Datalogics CTO, Matt Kuznicki, discusses content extraction from PDFs and Datalogics PDF Alchemist.

Published in: Technology
  • ➤➤ How Long Does She Want You to Last? Here's the link to the FREE report 
    Are you sure you want to  Yes  No
    Your message goes here
  • ★★ How Long Does She Want You to Last? ★★ A recent study proved that the average man lasts just 2-5 minutes in bed (during intercourse). The study also showed that many women need at least 7-10 minutes of intercourse to reach "The Big O" - and, worse still... 30% of women never get there during intercourse. Clearly, most men are NOT fulfilling there women's needs in bed. Now, as I've said many times - how long you can last is no guarantee of being a GREAT LOVER. But, not being able to last 20, 30 minutes or more, is definitely a sign that you're not going to "set your woman's world on fire" between the sheets. Question is: "What can you do to last longer?" Well, one of the best recommendations I can give you today is to read THIS report. In it, you'll discover a detailed guide to an Ancient Taoist Thrusting Technique that can help any man to last much longer in bed. I can vouch 100% for the technique because my husband has been using it for years :) Here's the link to the report ♣♣♣
    Are you sure you want to  Yes  No
    Your message goes here

Intelligent Content Extraction from PDFs

  1. 1. Intelligently Extracting Data from PDFs Presented by Matt Kuznicki Chief Technical Officer, Datalogics
  2. 2. Agenda • Technical Challenges in PDF Data Extraction • Key Considerations for Data Extraction • Use Cases • About Datalogics PDF Alchemist
  3. 3. About Me • Chief Technical Officer at Datalogics • Vice Chairman of PDF Association Board of Directors • Worked extensively with PDF for over 15 years • Active participant in the PDF standards community
  4. 4. Technical Challenges in PDF Data Extraction
  5. 5. Extraction: Technical Challenges • PDF is a page description language – elements typically have fixed position on a physical plane • Elements are not necessarily defined in order of appearance • Richer vocabulary for expressing elements than other formats • Structure and semantics of elements not commonly stated
  6. 6. At the time PDF was conceived in the 1990s, reliable rendering for human readers was an important issue • Focus was on retrieving the information needed to display and print pages for peoples’ use • Affordances to give content semantics came much later • Community has made great strides in allowing for machine interpretation, but proper use requires expertise in the domain • Structure and semantics are optional – usage is still rare • This is NOT a PDF specific issue PDF as Page Description Language
  7. 7. PDF as Page Description Language • PDF format most concerned with expressing exact visual representation • Elements are placed at fixed positions on virtual pages, in small discrete pieces • Not as fine-grained as individual dots in a raster file, but not as continuous content like most HTML • No guarantee of sentences or even letters grouped together to form whole words in a PDF data stream • Usually PDF files contain no information about how elements relate to each other
  8. 8. PDF as Page Description Language PDF pages often contain content that is a byproduct of breaking data into page-size chunks, such as: • Page numbers • Page headers and footers • Guides and information for printing These elements are not usually considered real document data, extracting these as content is usually undesired.
  9. 9. Elements and Ordering Small graphic elements can mean big extraction problems: • Contents of a PDF page can be specified in an order very different from how we read • Humans automatically see a page flow that is not always present in the PDF data stream • Words, images and other elements on a page may have the marks that constitute them spread far throughout the page marking stream • Without ordering information, flow of PDF content must be heuristically derived and is subject to differing interpretations
  10. 10. Richer Vocabulary For Elements PDF includes a richer way to express elements than most other languages: • Images can be in many different forms, including GIF, JPEG, PNG, JPEG 2000 and JBIG2 derived formats • Fonts can be in several forms, including OpenType, TrueType, Type 1, CFF, multiple master; or expressed in PDF element syntax • Text may be expressed in a way that includes Unicode information – or in one of hundreds of encodings – but no Unicode information is actually required • Rich transparency and blending model allows for complex element interaction • Content may be optionally present or absent from a page depending on a number of different triggers and conditions
  11. 11. Structure and Semantics Information on the structure and semantics of a PDF page is usually not present: • Lists are really just bunches of words and sometimes symbols humans interpret as bullets or delimiters • Tables are really just a series of lines and shaded boxes, and bunches of words, that humans interpret together as rows, columns and headers • Paragraphs are really just bunches of words positioned on a page in such a way that humans interpret them as sentences grouped together • Columns don’t exist in the PDF data stream, it’s just that us humans see elements grouped in a way that suggests columns
  12. 12. Structure and Semantics When creating PDFs, it is possible to include structure and semantics into the PDF: • Creating tagged PDF means the information for conversion is included directly into the PDF when it’s created – at the right time! • Easy to convert tagged PDF into other formats and to reflow • Not all tagged PDF is of good quality – and not all generators emit useful tagged PDF! Bottom line: you can’t count on getting PDF that has easily extractable content!
  13. 13. Key Considerations For Data Extraction
  14. 14. Extraction: Key Considerations • Content extraction means different things to different audiences • Know your audience and its goals • Different goals are best met through different means
  15. 15. Extraction: Different Meanings Let’s take a PDF that’s just one image of a scanned page:
  16. 16. Extraction: Different Meanings Let’s take a PDF that’s just one image of a scanned page: • Does extracting the content mean returning the image? • Does extracting the content mean OCRing the image and returning the text? If the PDF is an image and text underneath – is the content the image, the text, or both?
  17. 17. Know Your Audience’s Goals Different audiences have different needs: • Extraction for indexing or summarization typically requires a pure text stream of paragraphs • Extraction for loading contents into a database for machine learning typically does not need appearance preservation • Extraction for presentation on a different screen or medium typically means content order should be preserved but the appearance is expected to change
  18. 18. Different Goals, Different Means Different goals mean different trade-offs: • Indexing, machine learning, data mining – preservation of text and reconstruction of semantics most important • Reformatting for reflow or format conversion – balance between text preservation and appearance preservation needed • Reformatting for reliable viewing across devices – appearance preservation most important, text preservation secondary • Semantic reconstruction usually not required
  19. 19. Use Cases
  20. 20. Use Cases for Content Extraction • Conversion to HTML for viewing PDF without a PDF viewer • Converting PDF into a reflowable HTML representation • Extraction of PDF contents for machine understanding
  21. 21. Viewing PDF Without a PDF Viewer PDF extraction and conversion revolves around visual appearance: • Extract content and into a 1 to 1 analogue in a different fixed layout (HTML + SVG, raster image, print-out, etc.) • Convert extracted content into different visual primatives • Reliable viewing, but maintains disadvantages of PDF format This is the simplest and easiest way to convert PDF content for human reading – but doesn’t extract the content into a useful form for machines
  22. 22. Converting PDF Into Reflowable HTML PDF extraction and conversion balances needs of humans and machine understanding: • Elements are analyzed in page context and turned back into text flows, lists, tables, and other structured elements • Elements that can’t be expressed in HTML are usually rendered to allow proper viewing, at the loss of search-ability • Navigation elements – bookmarks, links – are converted into HTML equivalents for easy browsing • Pagination artifacts are discarded when possible Resulting HTML is reflowable and gives good document reading experience, but appearance typically changes somewhat to be more “HTML-ish”
  23. 23. Extraction of PDF Contents For Machine Understanding PDF extraction focused on text and structure: • Elements are analyzed in page context and turned back into text flows, lists, tables, and other structured elements • Text elements that can’t be expressed in HTML are usually left as text, sacrificing visual fidelity • Navigation elements – bookmarks, links – are converted so that automated processes can crawl these • Pagination artifacts should be discarded when possible
  24. 24. About Datalogics PDF Alchemist
  25. 25. Datalogics PDF Alchemist • Works on untagged PDFs – handles existing PDFs, does not require workflow changes or regenerating/reconstructing source PDFs • Turns placed words in PDFs back into reflowable text • Re-creates tables and lists from page content • Removes pagination artifacts such as page #s and running headers • Converts PDF into single-page HTML5 + CSS or into EPUB packages • Converts PDF forms into fixed-layout HTML forms for use in mobile environments
  26. 26. Summary
  27. 27. Extracting Content from PDFs Intelligently extracting content from PDF files requires: • Seeing pages in a way like a human reads them • Figuring our the logical structure of the pages • Putting text back together into text flows • Putting all these elements back together in the correct order • Compensating intelligently for differences between PDF and the chosen method of receiving content
  28. 28. Questions? Matt Kuznicki Chief Technical Officer LinkedIn: mattkuznicki Datalogics Inc. Twitter: @DatalogicsInc