Quality Assessment and Economic Sustainability of Translation Luigi Muzii

Caveat The content of this presentation treats quality from an “industrial” point of view

The content of this presentation treats quality from an “industrial” point of view

The starting point Translation is a cost Clients see translation as a commodity TSP’s are not paid for basic services Only VAS are worth spending Annex A in EN 15038:2006

Translation is a cost

Clients see translation as a commodity

TSP’s are not paid for basic services

Only VAS are worth spending

Annex A in EN 15038:2006

Translation as a cost Translation is a labor-intensive activity How should it be valued?

Translation is a labor-intensive activity

How should it be valued?

Doing more with less Customers’ expectations for multilingual information grow rapidly A translation that no one would read is not a deal Someone could even have a look at the job sometime Company resources and methods are dwarfing

Customers’ expectations for multilingual information grow rapidly

A translation that no one would read is not a deal

Someone could even have a look at the job sometime

Company resources and methods are dwarfing

A vicious circle Customers are willing to spare resources for new products, and want to pay less Vendors have smaller budgets Service providers are unwilling to work for less Vendors cannot satisfy customers Users see no benefit and are unwilling to spend

Differentiation How many translators and vendors claim their main differentiator is quality?

How many translators and vendors claim their main differentiator is quality?

The offer Quality varies widely with the scope of service If so many offer (excellent) quality, what differentiates competitors? No longer a selling point, but an expectation Deliver what promised and meet requirements

Quality varies widely with the scope of service

If so many offer (excellent) quality, what differentiates competitors?

No longer a selling point, but an expectation

Deliver what promised and meet requirements

Differentiator is in the client’s choice Nominally based on an array of criteria A unique operative point of comparison Quality is not an intrinsic value Unique Selling Proposition Clients assume equal levels of quality and service from any vendor

Nominally based on an array of criteria

A unique operative point of comparison

Quality is not an intrinsic value

Unique Selling Proposition

Clients assume equal levels of quality and service from any vendor

Does quality matter? Totally irrelevant from a sales perspective It is expected Is the quality you are selling the same quality your clients are buying?

Totally irrelevant from a sales perspective

It is expected

Is the quality you are selling the same quality your clients are buying?

Quality defined The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs An integration of the features and characteristics that determine the extent to which output satisfies the customer’s needs Compliance to generally accepted processes used consistently and constantly throughout an organization Business processes can be improved so that the product pass as it is Doing the right thing the right way the first time and every time

The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs

An integration of the features and characteristics that determine the extent to which output satisfies the customer’s needs

Compliance to generally accepted processes used consistently and constantly throughout an organization

Business processes can be improved so that the product pass as it is

Doing the right thing the right way the first time and every time

Cross views Quality is relative People can perceive different quality levels in the same product Quality is geared to constraints in requirements Different types of jobs need different quality requirements Different quality requirements lead to different expectations

Quality is relative

People can perceive different quality levels in the same product

Quality is geared to constraints in requirements

Different types of jobs need different quality requirements

Different quality requirements lead to different expectations

Does quality exist? Doing right the first time every time Preventing errors, not catching errors Dissociate language skills from productivity skills Education and training play different roles in building a translator’s profile Domain-specific knowledge are more important than language skills

Doing right the first time every time

Preventing errors, not catching errors

Dissociate language skills from productivity skills

Education and training play different roles in building a translator’s profile

Domain-specific knowledge are more important than language skills

Errors Important in the manufacturing industry Defective parts are discarded The production process is revised to avoid manufacturing of defective parts

Important in the manufacturing industry

Defective parts are discarded

The production process is revised to avoid manufacturing of defective parts

Quality in translation Little to do with errors Much to do with assumptions and goals Client-vendor mismatch Different approaches for different types of translation and needs Define process and responsibilities

Little to do with errors

Much to do with assumptions and goals

Client-vendor mismatch

Different approaches for different types of translation and needs

Define process and responsibilities

The origin of errors Disagreements from missing requirements When the project is over Require a lot of effort on both sides to fix Mainly stylistics or preferences Usually depend on translators disregarding style guide and glossary

Disagreements from missing requirements

When the project is over

Require a lot of effort on both sides to fix

Mainly stylistics or preferences

Usually depend on translators disregarding style guide and glossary

Translation quality assessment Based on strict correspondence between source and target texts Intensive error detection and analysis Totally uneconomic approach Considerable investment in human resources and time Technical translators trained by current university teaching methods and programs are generally not prepared to meet different quality criteria

Based on strict correspondence between source and target texts

Intensive error detection and analysis

Totally uneconomic approach

Considerable investment in human resources and time

Technical translators trained by current university teaching methods and programs are generally not prepared to meet different quality criteria

Major vs. minor errors Do minor errors exist? Error-counting assessment methodologies do not address all problems A translation may be free of errors and still unacceptable

Do minor errors exist?

Error-counting assessment methodologies do not address all problems

A translation may be free of errors and still unacceptable

Quality is free What costs money are the unquality things* Clients rely on the service provider to deliver a certain degree of intrinsic quality Many clients know almost nothing about the translation process Clients are not interested in any of the translator’s favored issues Few clients have the skills and competences to knowledgeably assess a translation Many c l i e n t s have pretty firm ideas about how much translation should cost * Philip B. Cosby from “Quality is Free: the Art of Making Quality Certain”

What costs money are the unquality things*

Clients rely on the service provider to deliver a certain degree of intrinsic quality

Many clients know almost nothing about the translation process

Clients are not interested in any of the translator’s favored issues

Few clients have the skills and competences to knowledgeably assess a translation

Many c l i e n t s have pretty firm ideas about how much translation should cost

Assessment models Le intérêt théorique des méthodes d’évaluation est inversement proportionnel à leur simplicité d’application.* Robert Larose * From Meta, XLIII, 2, 1998

Le intérêt théorique des méthodes d’évaluation est inversement proportionnel à leur simplicité d’application.*

Robert Larose

Death of the reviewer Are reviewers still needed? Do high-quality translations need further improvements? The translator has more information than the reviewer, when it should be the opposite

Are reviewers still needed?

Do high-quality translations need further improvements?

The translator has more information than the reviewer, when it should be the opposite

The process Teach how to avoid errors not how to catch them Try an ‘economic’ approach with the cost of errors Eliminate editors Introduce “project facilitators” Create a community Translators and a SME to answer questions All the tools to fulfill the project

Teach how to avoid errors not how to catch them

Try an ‘economic’ approach with the cost of errors

Eliminate editors

Introduce “project facilitators”

Create a community

Translators and a SME to answer questions

All the tools to fulfill the project

AOQ vs. AQL Average Outgoing Quality procedures best suited for small translation projects Sampling is non-destructive Lots are 100% inspected All defectives in rejected lots are replaced with good units Acceptance Quality Levels for tolerance and deviations Maximal percentage of non-conforming items to be considered as a satisfying process mean Acceptance sampling Lots exceeding deviations from AQL are rejected Agreed upon in a SLA

Average Outgoing Quality procedures best suited for small translation projects

Sampling is non-destructive

Lots are 100% inspected

All defectives in rejected lots are replaced with good units

Acceptance Quality Levels for tolerance and deviations

Maximal percentage of non-conforming items to be considered as a satisfying process mean

Acceptance sampling

Lots exceeding deviations from AQL are rejected

Agreed upon in a SLA

TQI vs. AQL TQI = Translation Quality Index A measurement of the numbers and types of errors Error definition, categorization, and ‘weight’ (severity) A statistical score Many a TQI depending on purpose 85% of an organization’s problems lie in its processes To know whether a process is quality-effective its results must be measured Objective, reproducible, and repeatable Quality before and after each stage Sampling implies/requires homogeneity (in batches)

TQI = Translation Quality Index

A measurement of the numbers and types of errors

Error definition, categorization, and ‘weight’ (severity)

A statistical score

Many a TQI depending on purpose

85% of an organization’s problems lie in its processes

To know whether a process is quality-effective its results must be measured

Objective, reproducible, and repeatable

Quality before and after each stage

Sampling implies/requires homogeneity (in batches)

Metrics Metrics measure how much a product meets requirements Metrics measure performance Metrics help identify specific performance-affecting problems Money is a measure that holds the same value for both parties Cost vs. revenues Quality of the finished product Number and magnitude of defects Quality of the process How reliable it is to produce quality products Likelihood of achieving quality in this deliverable Predictors of quality

Metrics measure how much a product meets requirements

Metrics measure performance

Metrics help identify specific performance-affecting problems

Money is a measure that holds the same value for both parties

Cost vs. revenues

Quality of the finished product

Number and magnitude of defects

Quality of the process

How reliable it is to produce quality products

Likelihood of achieving quality in this deliverable

Predictors of quality

The real cost of translation* Money is an objective metric It does not influence the system being measured Trade-off between time and performance c = cost t = translation e = error rate (%, 0 ≤ e ≤ 1) r = revision  f (t) a = accessories * From Salvador Aparicio i Paradell ’s address at the first international congress on specialized translation in Barcelona (March 2000) ) + r + a 1 + e c = t ( 1 - e

Money is an objective metric

It does not influence the system being measured

Trade-off between time and performance

Economic sustainability Economic sustainability of translation corresponds to allocation efficiency The best trade-off between requirements that cannot all be simultaneously met Quality must be proportioned to profit

Economic sustainability of translation corresponds to allocation efficiency

The best trade-off between requirements that cannot all be simultaneously met

Quality must be proportioned to profit

Value The consumer’s marginal utility How much something is worth to someone relative to other things Cost-determined for services The resources that went into making it Demand-determined for commodities Value = Benefits Price = Quality received Expectations

The consumer’s marginal utility

How much something is worth to someone relative to other things

Cost-determined for services

The resources that went into making it

Demand-determined for commodities

Generating value Translation belongs to business models emphasizing end value rather than use value Translation is a cost as it has no direct influence in potential profit margin The economic benefit of translation is invisible

Translation belongs to business models emphasizing end value rather than use value

Translation is a cost as it has no direct influence in potential profit margin

The economic benefit of translation is invisible

Respond to demands Focus on process Review and streamline operations Solve operational problems and automate business processes Expedite and reduce translation-related activities

Focus on process

Review and streamline operations

Solve operational problems and automate business processes

Expedite and reduce translation-related activities

Quality is never an accident It is always the result of intelligent effort. John Ruskin

It is always the result of intelligent effort.

John Ruskin