Taus qe summit dublin 2015 pemt analysis and valuation
•Download as PPTX, PDF•
1 like•244 views
PEMT Analysis and Valuation using an application developeed for SDL Trados Studio 2014 by Patrick Hartnett. Presentation delivered at the TAUS QE Summit 2015 in Dublin.
Recommended
More Related Content
What's hot
What's hot (17)
Viewers also liked
Similar to Taus qe summit dublin 2015 pemt analysis and valuation
Similar to Taus qe summit dublin 2015 pemt analysis and valuation (20)
More from Paul Filkin
More from Paul Filkin (9)
Recently uploaded
Recently uploaded (20)
Taus qe summit dublin 2015 pemt analysis and valuation
- 1. PEMT analysis and valuation TAUS QE Summit Dublin 2015 May 2015 Paul Filkin, SDL
- 2. 2 PEMT analysis and valuation Can you value it and how do you value it? How do we improve the processes around supporting post editors? What kind of data should we be trying to collect and why? What solutions or research are available today?
- 3. Day to day work is in a TEnT (Translation Environment Tool)
- 4. Which TEnT tools support comprehensive productivity metrics out of the box today?
- 5. 5 memoQ & Studio 2015 ○ Linguistic Quality Assurance – Only supports measurement of quality OmegaT ○ iOmegaT – Separate tool? – Difficult to use? – Comprehensive enough? – Not mainstream… although now integrates with GlobalSight Popular tools with “productivity metrics”
- 6. 6 What we think we need? ○ Post-Editing changes made to the translation/MT ○ Time taken to edit at segment level ○ Keystroke movements ○ Movement around the editing application ○ Resources used ○ Speed analytics ○ Quality of the translation/review ○ Client Rates (words/time) ○ Sensible/meaningful analytics ○ What else?
- 7. Introduction to the work of Patrick Hartnett via the SDL OpenExchange
- 8. 8 Studio Time Tracker Professional – SDL OpenExchange
- 12. 12 Working with Quality Metrics in editor
- 16. 16 Export data…
- 17. 17 Document Reports – Average Time/Segment
- 18. 18 Document Reports – Words per minute
- 19. 19 Document Reports – Quality Metrics
- 20. 20 Reports to do!! ○ Proof of work – From the Linguist point of view: they have proof of the work that they entertained and the time it took to achieve it; this might also help in renegotiating their contract in some cases where the issues are part of the initial planning. – From the LSP (or LP) point of view: they can explain clearly the issues along with detailed reports as to reasons why deadlines are not met; these reports should also form the basis for a structured review of what could improve in the next cycle/planning – From the Client point of view: they have clear overview of the issues with view to the details gathered at the translation level; this also builds on the trust aspect with the LSP as they are working on a much more transparent level ○ What else??
- 21. Copyright © 2008-2014 SDL plc. All rights reserved. All company names, brand names, trademarks, service marks, images and logos are the property of their respective owners. This presentation and its content are SDL confidential unless otherwise specified, and may not be copied, used or distributed except as authorised by SDL. Global Customer Experience Management
Editor's Notes
- These questions only scratch the surface of an industry conundrum around the management of PEMT for academia and commercial purposes.
- Quite literally this means the translator, or reviewer, is mobile and able to work anywhere. So if you wish to be able to collect metrics and measure productivity the solution needs to be simple and in the Translation Environment. Hooking them up to third party systems, or tools they don’t use on a day to day basis is not going to provide the volume or consistency of data you need. Can you value it and how do you value it? How do we improve the processes around supporting post editors? What kind of data should we be trying to collect and why?
- As far as I’m aware I’d say this is zero!! So this is where the problem starts. If the most common tools don’t support out of the box this then how can we get the volume of data needed from Translators/Reviewers that will support the analysis and valuation for productivity today?
- Quote from John Moran (http://www.welocalize.com/mt-translator-speed-interview-john-moran/ ) “Right now, the iOmegaT CAT tool and the utilities and analysis software that go with it require a good deal of technical ability to use. For that reason we currently only engage with one new client at a time. Our focus has been on well-known corporate or enterprise end-buyers of translation with complex integration requirements, such as with SDL TMS and SDL WorldServer. This has worked well so our next aim is to develop the system into a suite that is easier to use to widen the user-base to smaller LSPs and even translators, while listening to our small core of enterprise clients to improve the software for them too.”
- Notes from Patrick One of the main advantages of working with a tool like this is that it keeps a complete comparable copy of every single action applied that qualifies as an addition, deletion or adaption of any of the properties in the segments of the document that the linguist is working on. By comparable, we mean that a record is created that encapsulates not only the latest version of the segment after the changes have been applied, but also a cloned copy of the segment prior to those changes; so that it provides a complete cascading type of report representing each adaption to the segments as they are applied.
- Notes from Patrick The latest release implements a new feature that manages the quality metrics, which allows you to import some of the more known standards such as TAUS DQF and SAE J2450. This adds a new dynamic to the plugin with enormous benefit to how the quality of the translations from a document are measured in a revision cycle. The Quality Assessment reports generated from this data are useful in providing an overview on the commonality of the linguists errors, where they occurred, in what context and who (or what TM, MT etc…) applied them; keeping in mind that the type of reports generated from these assessments are relative to the what you are evaluating; meaning, it might also be applicable in assessing the translations from an MT engine in a certain domain. All of these are important characteristic as they provide the basis for calculating the production costs understanding not only what changed but also the time taken to make those changes. Because the data that is maintained with the plugin is so robust, some very interesting type of reports can be generated from this type of data that we will be working on next…
- Notes from Patrick The document overview is a very comprehensive overview of all the work entertained on the document from the linguist for single or multiple edit sessions. It compiles all interactions applied by the linguist while working on the document and represents those changes in a structured/ readable format. The cost of production is integrated with this report taking into consideration the association between the weight of the changes applied by the linguist and the language rates associated with the activity for that document or documents.
- Notes from Patrick Another very important aspect is the time taken and how the changes are applied by the linguist; for this purpose the tool also manages the production time and keystroke data. The type of production time data that is managed with the plugin: Opened and closed a document Document active on the screen and being worked on by the linguist; useful when the linguist is working on multiple documents contemporarily Entered and exited a segment The date/time spent typing or adapting the content in the segment; in this case the date/time is associated with the individual keystrokes The movement across the file
- Notes from Patrick Note: The decision to track the keystroke data is ultimately the decision of the user and ONLY tracks keystrokes from the Studio Editor window.
- Notes from Patrick Proof of work: something that I have noticed quite a lot recently as we are constantly evolving in more competitive environments, is when things go drastically wrong in a project. It happens to the most prepared professionals (PM, Linguists alike). Let’s examine a case where a linguist is required to use a specific MT engine, or TM or Terminology termbase as part of the contract and because of this -> the linguist spends twice as much time working on translating the content as required (due to common issues) and because of the deadline this also had an indirect degradation on the quality (simply because the linguist was forced to work in a certain way…) This is something that no one likes -> the unforeseen aspects of the project that turn your projects into a living nightmare for everyone involved. When deadlines are pushed to a certain degree, the involvement with your client is inevitable and understanding what went wrong in the process becomes paramount. Because this plugin provides you with unique insight to all interactions during the translation/revision life cycle, it can improve drastically your approach in explaining these scenarios. From the Linguist point of view: they have proof of the work that they entertained and the time it took to achieve it; this might also help in renegotiating their contract in some cases where the issues are part of the initial planning. From the LSP (or LP) point of view: they can explain clearly the issues along with detailed reports as to reasons why deadlines are not met; these reports should also form the basis for a structured review of what could improve in the next cycle/planning From the Client point of view: they have clear overview of the issues with view to the details gathered at the translation level; this also builds on the trust aspect with the LSP as they are working on a much more transparent level