Converting Unstructured Docs to        XML/DITA/ePubMark Gross          Linda Morone
Background of Data Conversion Laboratory 30 years of experience providing electronic document conversion services meeting ...
Serving All Industries              • Publishers              • Government              • Defense              • Life scie...
Serving a Broad Client Base                          (Confidential)   4
Converting Legacy Data … Is it Worth the Expense?             • Comply with regulations             • Match Industry stand...
Legacy Conversion: Fact or Fiction         Client’s Perception                    Reality     •    Painful Process        ...
So … Which Format do you ChooseePUB and Rendering-Focused DTDs               NLM and Publishing DTDs• Designed for e-reade...
The Story with ePub and Rendering-Focused DTDs       •   ePub is an emerging standard used for most eReaders       •   Mob...
Things to Keep in Mind When Converting                •   Smaller screen size                •   Large tables may not fit ...
OCR/Text Extraction          Pitfalls of Text Extraction          •   Special Characters          •   Emphasis          • ...
Handling of Objects Mid-Paragraph                                                 Converting exactly per source           ...
Math as Images – Changing Font Size Doesn’t Change Images                         (Confidential)   12
Unicode Symbols Will Adjust with the Font Size Change                          (Confidential)   13
Large Tables Table as Text (searchable but cut off)              Table as Image                               (Confidentia...
When Layout Matters        Testing Materials                         Poetry                            (Confidential)   15
When Layout Matters (cont’d)           Letter                               Recipe                          (Confidential)...
Some Notes on the Kindle           •   Designed for reading long documents           •   Designed for simplicity          ...
Glossary Definitions         iPad screenshot                         Kindle screenshot                           (Confiden...
Use of CSS “Float” Style       iPad screenshot                           Kindle screenshot                           (Conf...
Use of Borders         iPad screenshot                         Kindle screenshot                           (Confidential) ...
Color/Spanning/Large Tables          iPad screenshot                         Kindle screenshot                            ...
The Story with NLM and Publishing DTDs        •   Well-documented public domain standard.        •   Well-tested on a wide...
Choosing the Content to Convert         Which content will be auto-generated?                    •   TOC                  ...
Capturing Items as Multiple Formats                      Math as images and MathML                      Tables as images...
Determining Data Elements   Appearance Based:             Content Based:      •   Alignment                  •       <emai...
Granularity of Tagging: Front Matter                            (Confidential)   26
Granularity of Tagging: Back Matter            • Are the references Harvard or Numeric?            • Is the author name la...
The Story with DITA and Module-Based DTDs             • Allows for modularization of your content with Topics,            ...
What Makes DITA Conversions Difficult           “Getting there using DITA is like building with prefabricated modular     ...
Structuring a Book into Topics in DITA                   Book 3                  Reference 4                              ...
Further Complications in DITA Conversions              •   There’s the usual conversion issues                   – Accurac...
Overview of Typical DITA Technical Conversion Issues             •   Architectural constraints of DITA – the square pegs  ...
Square Peg 1 - Task / Procedure Authored As a Table     Issue:         Tasks are done as tables rather than numbered lists...
Square Peg 2 - Multiple Steps In A Single Task   Issue:        Only one set of steps is allowed in a single task topic. Wh...
Square Peg 3 - Irrelevant Cross-References      Issue:          Conversion to DITA may make some source cross-references i...
So … Maybe You Shouldn’t Bother Converting Your Content?         •   It seems like such a pain to go through all the old l...
In Reality … Converting Your Content is Worth the Bother    • Throwing it out and starting over is an expensive option    ...
Conversion Scope Options   Option 1: Convert nothing   • No conversion costs                                           2  ...
What to Convert, and in What Order       •   Categorizing            – Active documents in good shape            – Active ...
Closing Thoughts     • Know the scope of what you want to accomplish         – Are you trying to get eBooks quickly, or ar...
Questions...                                                      & Answers   Data Conversion Laboratory             Mark ...
Upcoming SlideShare
Loading in...5
×

Converting Unstructured Docs to XML/DITA/ePub

929

Published on

DCL's Presentation for LavaCon 2011

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
929
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Converting Unstructured Docs to XML/DITA/ePub

  1. 1. Converting Unstructured Docs to XML/DITA/ePubMark Gross Linda Morone
  2. 2. Background of Data Conversion Laboratory 30 years of experience providing electronic document conversion services meeting the needs of technology…today & in the future • More than 1 billion pages converted to date • US Based project management team • Global capabilities • Transform legacy & future documents • From any format to any format • Specialize in complex projects • Identify redundant data for content reuse • Employ a proven automated process • Quality Assurance service is standard in all projects • Additional services include consulting, composition & transcription & translation (Confidential) 2
  3. 3. Serving All Industries • Publishers • Government • Defense • Life sciences • Automotive • Aerospace • Heavy and Industrial Equipment • Financial Services • Manufacturing • Computing • Utilities • Semiconductors • Telecommunications (Confidential) 3
  4. 4. Serving a Broad Client Base (Confidential) 4
  5. 5. Converting Legacy Data … Is it Worth the Expense? • Comply with regulations • Match Industry standards • Meet customer expectations & needs • Support internal departments • Expand into new markets • Multi-purpose content (Confidential) 5
  6. 6. Legacy Conversion: Fact or Fiction Client’s Perception Reality • Painful Process • Expertise & Planning • Complex • QC & Automation • Expensive • Guaranteed Results • Drain on Resources • Low Costs (Confidential) 6
  7. 7. So … Which Format do you ChooseePUB and Rendering-Focused DTDs NLM and Publishing DTDs• Designed for e-readers & mobile • Support traditional publishing devices • Flexible open standard• Freely available • Freely available• Open standard • Human-readable format• Adaptable to – Books DITA and Module-Based DTDs – Documents • Designed for multi-purposing and – Manuals content reuse – User guides • Topic based & modular• Support for print publishing • Supports requirements is limited – Multiple variants – Multiple languages – Context independent content (Confidential) 7
  8. 8. The Story with ePub and Rendering-Focused DTDs • ePub is an emerging standard used for most eReaders • Mobi is also a large player, proprietary to Amazon Kindle • ePub is an evolving standard • ePub is supported differently by different eReaders • There are no “Silver Bullets” • eBooks are publications and need care in their production • Not just novels; recent DCL survey shows 75% will be using eBooks for complex materials (Confidential) 8
  9. 9. Things to Keep in Mind When Converting • Smaller screen size • Large tables may not fit • Not all Character Sets supported by all devices • MathML not currently supported (Confidential) 9
  10. 10. OCR/Text Extraction Pitfalls of Text Extraction • Special Characters • Emphasis • Ligatures • Hyphens – Soft and Hard (Confidential) 10
  11. 11. Handling of Objects Mid-Paragraph Converting exactly per source may lead to problems … (Confidential) 11
  12. 12. Math as Images – Changing Font Size Doesn’t Change Images (Confidential) 12
  13. 13. Unicode Symbols Will Adjust with the Font Size Change (Confidential) 13
  14. 14. Large Tables Table as Text (searchable but cut off) Table as Image (Confidential) 14
  15. 15. When Layout Matters Testing Materials Poetry (Confidential) 15
  16. 16. When Layout Matters (cont’d) Letter Recipe (Confidential) 16
  17. 17. Some Notes on the Kindle • Designed for reading long documents • Designed for simplicity • Has some features that others don’t • But also missing some features that others have • Therefore, need to design the conversion differently (Confidential) 17
  18. 18. Glossary Definitions iPad screenshot Kindle screenshot (Confidential) 18
  19. 19. Use of CSS “Float” Style iPad screenshot Kindle screenshot (Confidential) 19
  20. 20. Use of Borders iPad screenshot Kindle screenshot (Confidential) 20
  21. 21. Color/Spanning/Large Tables iPad screenshot Kindle screenshot (Confidential) 21
  22. 22. The Story with NLM and Publishing DTDs • Well-documented public domain standard. • Well-tested on a wide variety of materials; designed for complex publishing. • Originally designed with NIH support for Scientific, Technical, and Medical (STM) publications. • Extended to be robust for many more uses; widely used in non-STM areas. • DocBook and PRISM are other standard DTD’s; each with its own strengths – all designed for “print” publications. (Confidential) 22
  23. 23. Choosing the Content to Convert Which content will be auto-generated? • TOC • Index • Labels • Titles • List of Table, Figures, etc. (Confidential) 23
  24. 24. Capturing Items as Multiple Formats  Math as images and MathML  Tables as images and XHTML <disp-formula id="FD1"> <mml:math id="M1" display=block> <mml:semantics> <mml:mrow> <mml:mi>L</mml:mi> <mml:mo>=</mml:mo> <mml:mo>∑</mml:mo> <mml:mrow> <mml:msub> <mml:mrow> <mml:mi>l</mml:mi></mml:mrow> <mml:mi>i</mml:mi></mml:msub> <mml:mo>/</mml:mo> <mml:mi>N</mml:mi></mml:mrow> <mml:mo>.</mml:mo></mml:mrow> </mml:semantics></mml:math> </disp-formula> (Confidential) 24
  25. 25. Determining Data Elements Appearance Based: Content Based: • Alignment • <email> - @ • Placement • <uri> - www • Point size • <degrees> - PhD, MD, BA • Font • <fig> - Figure, Illustration, Chart, Scheme (Confidential) 25
  26. 26. Granularity of Tagging: Front Matter (Confidential) 26
  27. 27. Granularity of Tagging: Back Matter • Are the references Harvard or Numeric? • Is the author name last/first or first/last? • What is the placement of the year within the citation? • Is a comma or period used after the author names? (Confidential) 27
  28. 28. The Story with DITA and Module-Based DTDs • Allows for modularization of your content with Topics, and easy re-use in multiple outputs • Pre-packaged & ready to use XML (almost) • Ready-to-go for techdocs (mostly) • Infrastructure included - taxonomy (DTD and schema); printing stylesheets; lots of tools • Printable with standard tools • Extensible with specializations • Further specializations for publishing, testing, and other specialized areas • Content-based • What do you when things don’t fit (Confidential) 28
  29. 29. What Makes DITA Conversions Difficult “Getting there using DITA is like building with prefabricated modular components that can be quickly assembled into a suitable structure.” - Doug Henschen, intelligententerprise.com • DITA is a conceptual departure from linear information – and is difficult for many to get used to • Turns the traditional book into a collection of Topics • Topics can be thought of as interchangeable parts – to be reassembled in multiple ways – to be repurposed for multiple outputs – to be reused across multiple products • …but your documents weren’t likely to have been designed to do this. (Confidential) 29
  30. 30. Structuring a Book into Topics in DITA Book 3 Reference 4 Book A Book 1 Concept 1 Reference 1 Concept 4 Reference 1 DITA Content Reference 2 Task 3 Concept 3 Management System Concept 5 Reference 3 Reference 1 Task 2 Task 1 Concept 2 Concept 1 Reference 1 Task 2 Reference 5 Concept 2 Reference 2 Task 1 Book 4 Concept 3 Reference 3 Book B Concept 2 Concept 4 Reference 4 Concept 1 Task 1 Concept 5 Reference 5 Book 2 Reference 1 Concept 3 Task 1 Task 1 Concept 1 Task 2 Task 2 Concept 2 Reference 2 Reference 3 Reference 2 Concept 5 Task 3 Task 3 Task 3 Task 2 Task 1 “Getting there using DITA is like building with prefabricated modular Reference 5 components that can be quickly assembled into a suitable structure.” Concept 2 – Doug Henschen, intelligententerprise.com (Confidential) 30
  31. 31. Further Complications in DITA Conversions • There’s the usual conversion issues – Accuracy of the transferred text – Tables – Math – Special Characters • There’s also the structuring issues – Identifying topics – Identifying reusable content • And the people issues – Deciding what needs re-authoring – Getting used to a new “document” paradigm – Getting rugged individualists to collaborate more (Confidential) 31
  32. 32. Overview of Typical DITA Technical Conversion Issues • Architectural constraints of DITA – the square pegs – Multiple steps within a single task topic – TaskProcedure authored as a table in the source – Presence of untitled tasks/topics in the source – References to page numbers (irrelevant cross-references) – Having more than two levels of steps • How your rendering system will handle XML – Figures – Steps • Other conversion considerations: – Hierarchy in Map Files – Metadata in Map Files and Topics – Index Terms – Conditional Text – Glossary Terms – Content Terms (Confidential) 32
  33. 33. Square Peg 1 - Task / Procedure Authored As a Table Issue: Tasks are done as tables rather than numbered lists. If there’s no clear consistent pattern, then automated conversion keeps the tables as tables, and steps are not tagged as steps. 1 Overview In general, backup and recovery refers to the various strategies and procedures involved in protecting a system against data loss. 2 Backup strategy and A backup is a copy of key files. Files included frequency in the backup are: • A logical backup of the database 1. Key system files • Network files • Timezone 2. Configuration files … (Confidential) 33
  34. 34. Square Peg 2 - Multiple Steps In A Single Task Issue: Only one set of steps is allowed in a single task topic. When a task has two sets of steps within a topic, such as for two different scenarios, only one of the scenarios can be tagged as <steps> as per the DTD. Example: Replacing an XYZ Module Use this procedure to replace an XYZ module Remove XYZ Module 1. Loosen the screws. 2. Disengage the ejectors 3. Pull the module straight out Insert Replacement XYZ Module 1. Align the module. 2. Insert the module, pressing in firmly 3. Engage the ejectors 4. Securely tighten the screws (Confidential) 34
  35. 35. Square Peg 3 - Irrelevant Cross-References Issue: Conversion to DITA may make some source cross-references irrelevant. For example, assuming all empty chapter headings are dropped, a reference to a chapter is no longer valid. In these cases, a <required- cleanup> tag is inserted to flag these occurrences for clean-up. See Chapter 1, Introduction on page 2 Would be tagged as: See <required-cleanup><xref href=”chap1”> Chapter 1, Introduction</xref></required-cleanup> NOTE: Hard-keyed page numbers are typically dropped from the cross- reference string since they are no longer relevant in DITA. (Confidential) 35
  36. 36. So … Maybe You Shouldn’t Bother Converting Your Content? • It seems like such a pain to go through all the old luggage in the attic. • There is always a need for some rewriting - few writers have the clairvoyance to author content with the intent that be converted in the future – might as well rewrite it all. • My writers aren’t very busy right now anyway. • It’s more fun and seems like less trouble to author anew. (Confidential) 36
  37. 37. In Reality … Converting Your Content is Worth the Bother • Throwing it out and starting over is an expensive option – In DITA, rewriting at $25/page vs. converting at $3-$4/page – The hidden costs of redoing index entries, links and other features you’ve built in – The hidden cost of reviewing, reproofing, and recertifying it all • It’s usually easier to use what you have as a base, and convert over – Needs planning – Needs time • Planning for a good conversion experience – Which content will you need? – Which content is worth converting? – Which content is suitable for re-use in multiple places? – What tools are available? – How to specify the conversion to get it right? – When do you start all this planning? (Confidential) 37
  38. 38. Conversion Scope Options Option 1: Convert nothing • No conversion costs 2 • Delayed ROI 1 Option 2: Convert everything cost • High conversion costs • Reduced ROI 3 Option 3: Convert ‘frequently used’ documents • Some conversion costs • Maximized ROI time (Confidential) 38
  39. 39. What to Convert, and in What Order • Categorizing – Active documents in good shape – Active documents that need a lot of work – Somewhat inactive document that will likely be retired – Archival materials • Prioritizing – Documents that are most used – Documents that are customer favorites – Documents with longest product life – Start with most recent documents and go back • Identifying the process – Can be converted as is – Can be converted with some work – Needs to be rewritten – Don’t convert – just keep archival copies (Confidential) 39
  40. 40. Closing Thoughts • Know the scope of what you want to accomplish – Are you trying to get eBooks quickly, or are you changing your publishing process – Are you moving everything, or will a phased approach work – Will your content work naturally with the selected DTD • Start the conversion process early – Shifts the critical path; speeds the process; reduces cleanup – Organizing early lets more of the work be done by the content owners – eases the training and change acceptance burdens – setting up collaborative teams sets the tone and allows one to “divide and conquer” • Converting legacy data is not trivial – …but faster, safer and less expensive than rewriting – Each DTD has special considerations to be taken into account – Much can be automated, but it needs planning (Confidential) 40
  41. 41. Questions... & Answers Data Conversion Laboratory Mark Gross, President 61-18 190th St., 2nd Floor mgross@dclab.com Fresh Meadows, NY 11365 718-307-5711 Telephone: (718) 357-8700 Linda Morone, Sr. VP of Sales & Marketing Fax: (718) 357-8776 lmorone@dclab.com Web: http://www.dclab.com 718-307-5728 (Confidential) 41
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×