Ilse Depraetere (Ed.), Perspectives on Translation Quality. Berlin: De Gruyter Mouton, 2011. 273 pp. ISBN 978 3 11 025984 1.

Reviewed by Joanna Drugan (University of East Anglia, UK)

This collection of eleven articles aims to avoid the academic-industry divide, which seems particularly acute and persistent on the topic of translation quality. For Jody Byrne (2007:2), for instance, the tendency has been "to regard translation errors solely from the point of view of academic studies and translation pedagogy, completely shut off from professional practice". The volume's editor, academic Ilse Depraetere, emphasises in her Introduction that it offers a  "different perspective", by including articles written by both university teachers and industry professionals. Practising translators make a further contribution by translating four of the articles into English, the language chosen for the publication.

The articles are presented in four sections, focusing on "the translation training context", Machine Translation (MT) evaluation, Quality Assurance (QA) in the translation workflow, and "domain-specific" translation, legal and literary texts being the two domains under discussion.

In the first paper, Anne Delizée outlines an evaluation method devised at Lille 3 University (France) for assessing Masters-level students of translation. The context and objectives of the assessment are outlined with admirable clarity, as are the details of its implementation at Lille 3 (e.g. the weighting assigned to different features). Less clear are some of the assumptions and terms. Students are assessed, for instance, on "pragmatic texts", following Delisle's 1980 definition, translated here as "texts whose basic purpose is to provide information and in which the aesthetic aspect is not dominant". [This, of course, begs the question: what texts do not have as a basic purpose to "provide information"?] Even if Delisle's classification is understood and accepted by the "layman", stipulated by Depraetere as part of the intended readership for the volume, Delizée's own category seems broader, and perhaps even contradictory: she includes "journalistic" and "commercial" texts, and at least some of the students were apparently localising websites, where "aesthetic aspects" would seem likely to dominate. Some confusion may be attributable to the fact that her arguments appear in translation from French, and here we come up against one of the inherent challenges in writing about translation quality. It takes some courage to enter the fray, when every (inevitable) typo or translation error in your own work is contrasted with your focus on the lofty goal of raising or measuring quality in that of others. But when you read such sentences as "Or, on the contrary, is it stylistically heavy or awkward and have lexical infelicities or inappropriate borrowings from the SL?", it is hard not to wonder why they were not checked before publication, perhaps using the translation quality assessment approach outlined by the author. The French context from which Delizée writes may also explain some odd assumptions (to this UK-based reader, at least), such as the statement that students should be working at "± 1 pg A4/hour" to be considered "profitable in the profession". How would this translate (sorry…) to cases of software or website localisation, for example? Are revision and other aspects of producing quality translations included in this work rate? And how many (source? target?) words are represented by "± 1 pg A4"?

The second article, by Depraetere herself and Thomas Vackier, is clearly linked to the first in its focus on translation evaluation in the academic context, again at Lille 3, sharing the same Masters programme, assessment approach and student translations as data. The novelty of their article lies in their contrasting of this academic approach to the application of an industry tool for translation quality assessment, QA Distiller. The authors believe this test offers benefits for both academics and the industry. Those training translators can check whether their assessment methods reflect industry standards, while the industry is keen to know whether the degree to which a translation suffers from "formal" errors (such as incorrect number formatting) might predict its overall translation quality. The test (on a corpus of 17 student translation projects, all from Spanish to French) found that "50% of the cases [showed] a rather strict correspondence between formal errors and errors in meaning". This leads the authors to conclude that further application of the model to a larger corpus would be worthwhile, to test the "interesting hypothesis" that it may be possible to identify which translations are most at risk of serious errors purely by automatic testing of "formal" aspects. If this were established, it might provide a useful and cheap way to decide which translations ought to be subject to more rigorous (human) quality assessment. The authors present clear illustrations throughout and point out valid criticisms of the automated tools, such as false positives in error logs.

In contrast, the third article, by Carmen Núñes-Lagos and Nathalie Moulard, returns the focus to academic concerns alone, considering "number and gender agreement errors" in student translations from Spanish into French. Again, the same evaluation scale and approach (as outlined in the first two articles) is applied to the translations, but this time, the authors focus on a specific error type. They first outline their approach, categorise the error types, with copious examples from the student corpus, then indicate some "points of focal attention in the training of future translators (from Spanish to French)". It is particularly hard to see why this article should have been written in English. Anyone training translators to work from Spanish to French is more likely to be a mother tongue speaker of French, for a start, and may therefore not be able to access this information if not also fluent in English; but more importantly, the presentation of the illustrations is necessarily horribly clumsy. Back translations into English are provided and welcome, but these cannot, of course, show the "number and gender agreement errors" which are the focus of the article, because such agreements don't exist in the same way for English. Thus the authors are forced to explain each example at some length: "In French, grain, when referring to coffee must be used in the plural, whereas in Spanish a singular form is used. In Spanish, en grano refers to the substance (coffee beans) and is in contrast with en polvo, grind coffee [sic]. French requires coffee to be referred to as a whole composed of units, i.e. beans. In other words, the translation does not capture the generic meaning in an appropriate way." 

The last article in this section, by Christopher Gledhill, investigates a "lexicogrammar" approach to checking translation quality in the academic context. His purpose is to "raise a number of research questions", though he does state his belief that these should interest anyone in the translation "business". Gledhill's main interest is in whether the lexicogrammar approach might help in identifying which of two translations is best. He begins with a definition of the basic theoretical assumptions underlying his approach then outlines his test data. The long-suffering students at Lille 3 are once again pressed into service. This time, they were asked to translate text from the website of Dassault aviation, from French to English, without access to "dictionaries or internet" (the rationale for this deprivation is not explained). Their translations were then compared to the official English translation as published on the Dassault website. Several detailed tables present the source text and both translations. By comparing these, the author identifies several features of a successful technical translation from French into English (e.g.  it should use "clauses and gerunds to translate lexical verbs and nominals"). He then aims to "show that one formulation is better than or at least more appropriate to the general lexicogrammar of the English language" by comparing two translations ("push back the limits" vs. "push the envelope") over six pages of text. While recognising that this "forensic" approach to quality analysis is unlikely ever to be commercially viable or appropriate for entire texts, his hope is that it "reminds us why the act of translation is at the same time so difficult, mysterious and ultimately, so fascinating".

Part 2 contains two complementary articles on machine translation evaluation. In the first, the volume editor Depraetere outlines four techniques for evaluating MT output. She tests these techniques on a small corpus of 2,250 source words translated using both a rule-based MT system (Systran 6.0) and a statistical MT system (Language Weaver), though recognises that the small scale of her tests does not allow her to formulate any "definitive conclusions about quality differences between the MT systems used". Rather, she is evaluating the evaluation methods. Her test nonetheless finds that results of human evaluation of MT output adequacy and fluency correlate; and that the results of human evaluation and post-editing evaluation technique are also linked. The author is careful to link her test to potential applications or findings relevant for real-world contexts. This provides a neat link to the next paper, by Nathalie De Sutter, proposing that MT evaluation be linked to post-editing effort. She measures the time spent by human translators on post-editing a test translation from English to French produced by rule-based and statistical MT systems. The human translators in question being, of course, the Lille 3 Masters students, 22 of whom carried out the post-editing. A further ten Lille 3 students rated the quality of the MT output using the DARPA five-point evaluation scale. The two sets of data were then compared, with De Sutter concluding that "edit distance" might be used to rate MT output quality in absolute figures, as the ratings obtained are "very similar" to those obtained by human evaluation.

So far, so academic: even if there are attempts in the first two sections to integrate professional concerns, none of these articles would be out of place in a traditional collection of academic essays on the topic. Part 3, however, shifts the focus to "professional testimony". First, Nancy Matis provides a descriptive general account of how quality assurance is typically integrated in the translation workflow. She includes a "General checklist", "Evaluation form" and several useful screenshots of the processes she describes. While no doubt reflective of practice in much of the industry, some of her claims seem overstated or generalised (e.g. that QA usually takes place at the end of the workflow). She does acknowledge this in the Introduction, stating that her comments are limited to "simple translation projects". Many of her comments thus seem somewhat removed from industry realities: e.g. "large projects usually require the involvement of several people". The move away from academic concerns is evident in the lack of (recognised) definitions. Although the author distinguishes linguistic from technical QA, as is common in much of the industry, there is no explanation of (say) the difference between quality assurance, control and evaluation (not all of which come at the end of the workflow). Musing on the very purpose of QA, the conclusion seems almost tautological: "If QA is performed by competent staff, it will generally help to improve the quality of the project, or at least guarantee that quality has already been achieved, which is obviously also essential in any quality process".

The third contribution by Depraetere, this time with Antonia Debove and Sabrina Furlan, offers a detailed comparison of five industry QA tools. They also consider the timing of QA in the workflow and acknowledge real-world constraints such as budgets and deadlines; for the first time in the volume, return on investment is mentioned. Very detailed tables, lists and screenshots offer helpful illustrations of their comparisons. Once again, a small translation corpus from English to French, this time with errors deliberately seeded in it, is used to test and compare the tools, with the authors concluding that all five are easy to work with and highly customisable, even for novice users, though they identify Xbench as the most user-friendly. They consider how far "independent translators" (i.e. individual freelance workers) might benefit from use of the tools, identify the dangers of false positives in the results, and conclude that the tools do enhance consistency and safeguard formal aspects of translation quality.

In the last article in this section, José Luis Vega Expósito, of the Directorate-General for Translation at the European Commission, gives a detailed overview of how the Spanish Department manages quality through the use of electronic tools, particularly translation memories. Specific illustrations from Spanish, French and English are provided, along with screenshots of the tools and interfaces familiar to EC translators. The driving aim is to make "administrative and legal language clearer, coherent and stable".

Part 4 begins with a related topic: quality issues in legal translation. Authors Patricia Vanden Bulcke and Armand Héroguel consider what constitutes a legal text, strategies used to translate such texts and criteria used by legal translators to select one strategy over another. Detailed examples from Belgium, France, Spain and the Netherlands are provided in Dutch, French and Spanish, though often, frustratingly, without back translations, making it difficult to judge, or even follow, the authors' arguments. From the dense legal concepts, it is quite a leap to the last article in the collection, on the problem of self-assessment in literary translation. In this, published Dutch to French translator and teacher Olivier Vanwersch-Cot, outlines the commissioning, translation and revision/proofreading processes, offering his own reflections on good practice (e.g. "I try not to bother the author. When the Dutch text seems obscure to me, I try to find out by myself the meaning of what I do not understand"). Specific illustrations of translation difficulties are again undermined by the decision to publish in English rather than in French or Dutch, as would seem logical. For instance, a discussion of whether or not to use the second form of the conditional perfect and references to the French grammar bible, Le Bon Usage, are likely to be meaningless to readers without French as one of their languages.

Overall, this diverse collection casts fresh light on a longstanding concern in both training and the translation profession. The attempt to bring together professional and academic concerns highlights both the strengths and weaknesses of each approach, offering ample ground for further fruitful study. The participation of this range of writers demonstrates encouraging goodwill on both sides to move the debate forwards, at a point when developments in technology and increased availability of translation data present us with exciting new opportunities to do so. Perhaps even in languages other than English?

References

Byrne, J. 2007. "Caveat translator. Understanding the legal consequences of errors in professional translation". The Journal of Specialised Translation, 7: 2-24.