Finding translation quality in the dark

There is a chicken and egg problem when it comes to translations. Those who are capable of judging the quality of a work, are also the ones who don’t have any particular need for the translation. Meanwhile, the people who must rely on a translation, are also the ones least able to decide whether a translation is good or not. Is there any way that we can close this gap of knowledge?

Welcome to applied epistemology!

But first, the problem needs to be clarified somewhat. Let’s assume that there exists in the word “good” translations as well as “bad” ones. Now, all translations exist on a spectrum from good to bad, so there’s really good ones, really bad ones, and many in between.

I’ll have to admit that this notion of “good” and “bad” are subjective. My judgment of “good” might be someone’s “bad”. We’re never going to be able to get around this fact, so we’ll admit it up front. However, this does not imply that we can’t be at least somewhat objective about it. I’d argue that while we might not agree on the specifics, we can probably agree on at least some basic criteria we’d like in a “good” translation. Perhaps:

  1. The translation accurately conveys the information of the original (for some notion of ‘accurate’)
  2. The translation reads in an understandable way, suitable for the situation (you might say ‘natural’ but sometimes you don’t want ‘natural’ per se)

I think, for the absolutely most basic of requirements, we’d need at least those two. Some notion of transferring the information over properly, and being able to be understood by the readers properly. Now, you might feel there are other criteria that’s important too —- perhaps conveying the intent of the work is important, or the specific presentation style, or how fast it’s produced, I can think of all sorts of situations when one or more of those might matter, but there are times when they don’t too.

So, sticking to the 2 most fundamental criteria I can think of, there’s one thing to note. #2 is something anyone that the translation is targeted at can make judgments on. For example, a native English speaker is perfectly capable of reading a translation, and making a judgment on whether it’s a good translation with respect to how understandable it is in English. If the translation is riddled with grammatical mistakes, if it makes no sense, if the writing is horrific, the reader can figure that out with no help

However, that same English speaker will be unable to make many judgments about #1. I don’t say it’s impossible, because a clever reader can get hints. If a line is inconsistent with the whole flow of the text, if it reads fine but still “sounds weird”, that might be an indication that there’s something going wrong with #1. However, it’s hard to be sure of it since the line might really be translated properly, and it’s the original that’s gone insane.

How do we solve this problem? Well, the only people who are capable of making judgments about #1 are people who know both languages. They can look at a pair of lines and come out with a sense of how “close” the two are. This is, of course, a qualitative judgment since it would take into account #1, #2, and whatever other criteria that person uses to judge translations. For many, it’s also just an instinctual thing, they don’t go through a big list of “is it clear? is it accurate? does it do this?”

Now, anyone that knows both languages well enough can make some form of judgment on the quality of a translation with respect to both criteria I’ve put forward. However, I’d like to suggest that even among those people, there’s something of a ranking, where other translators would have a greater weight of opinion than the average bilingual reader.

The advantage that other translators would have is that they are exposed to the issues surrounding translation much more. That would imply that they have a better sense of what’s possible in translation. If they read a translation that’s more clever, more beautiful, more skilled than what they can produce themselves, they’d know it. Same for when they encounter work that’s less skilled than what they can produce.

The effect you get here is that instead of being able to just say “this translation is good enough” or not, they have the ability to put something in relation to other works. It is the ability to judge a translation that reads somewhat strangely, but came from such a difficult to translate work that it can be considered high quality. It’s the ability to appreciate the finer details that go into a translation that other people might not think about.

In fact, perhaps the best way to judge the quality of a translation for a non-bilingual reader is to compare multiple translations of the same work. The differences are always striking and show just how much room can be between translations.

But still, through all this we have only 1 person’s opinion, and everything above is still subjective. How do we move beyond that? The answer might lie in the notion of ‘social construction.’ Social constructs are things that exist in society that are “invented” by people and exist only because we all behave to maintain it. The common example is fiat money, the symbols printed on a $100 bill and a $1 bill differ only slightly, it’s the same paper and ink, but we behave as if the $100 is worth much more. The social institution of paper money is ingrained into society.

I’m not suggesting that we go out and build a social institution. Instead, I’m suggesting we adopt pieces of existing ones. In the sciences, there’s the peer review system where peer experts read and critique each other’s work, then over time experiments are replicated and eventually a paper becomes accepted into the growing body of knowledge that is “science.” In the old European guild system, a journeyman applying to enter a guild needed to produce a masterpiece that the other masters in the guild needed to accept. There’s the circles of critics, from food, music, to the arts. All of these have “experts” who in some way or other make comment on the quality of a given work.

What I’d like to see is a climate of where the work of translators are looked at by other translators and commented on. While there would be a number of opinions about a given translation, a pattern should arise when you see that a number respected translators say that another translator’s work is good. How a reader might want to interpret those ratings is their freedom, but now they have more hints to base their decision on.

Note that I mentioned “respected” translators. Reputation is extremely important to this sort of system, since it’s a web of references and trust. Anyone can say anything they want on the Internet, but nothing speaks as loudly as a long history of respected works. Anonymous translators may be amazing or they may be incompetent, the problem is that you’ll never know what you’ll get from project to project.

Normally, in a larger community, this sort of system is already informally in place. Word gets around that a certain fansub group has good translations, and so on. It’s not really the case so far in visual novels. The community of translators is smaller, the pool of works that are released are tiny, and new works take a long time to come out. Because we’re so small, no one wants to duplicate effort, so it is extremely rare for competing translations to come out.

For the moment, all we have is the whispers of the hearsay on forums, IRC and friends, maybe that’s enough. Perhaps we should be doing something more…


  1. 09/26/2008 10:14 AM | #

    Agi, to prove his worth as a translator, that person must be able to translate Nasu-speak with some profiency, with the two guidelines for good translation used. If he makes more than an arbitary amount of mistakes (or even finishes the whole thing at all), we give the guy an email and an interview to assess his character.

    I mean, that’s how I would test would-be translators. Fansubs or otherwise.

    Nasu makes people rage.

    And as for people to peer-review fellow translators-to-be? I dunno man. Finding good translators with good personalities are very hard to find.

Commenting is closed for this article.