Google’s Love to Machine Translation Is beyond my Understanding

Svetlana Gladkova,

on November 11, 2008,

Honestly, I did not want to write about Google Reader team proudly reporting on making the world wide web truly worldwide by offering the functionality to translate any feed you read into your native language (on condition that your native language is among those supported by Google’s machine translation system). Really, I thought I have said enough about poor quality of machine translation overall and Google translation in particular as well as about the dubious practice of using volunteers for their own translation projects instead of paying to experienced translators to localize their multiple products into multiple languages to ensure acceptable quality of translation instead of what we now have with Google’s products often used in English by many people knowing Google’s native language well enough not to have to deal with the poorly translated versions in their own native languages.

But still it looks like Google is very enthusiastic about automated translation and can’t resist trying to help you translate everything you want into any language you want for free using yet another Google’s tool. Unfortunately for Google as a person with 7 years of translation experience myself I can tell that you will hardly ever find a translator who will agree that machine translation can be useful for anything, even to understand the general meaning of the text simply because meaning is exactly what a machine-translated text lacks.

Yet we see another experiment from Google promising to help everyone in the world understand everyone else, no matter what combination of languages the two persons speak. Yesterday Google officially added translation functionality to Google Reader. This functionality allows you to read any feed of your choice in your native language (or another language you want it to be displayed in). In the settings of any feed you can now choose to have this feed automatically translated into your language.

First of all, I have my doubts about the necessity of this functionality simply because it is pretty difficult to figure out how we are supposed to discover the feeds in other languages that you might be interested in reading. Are we supposed to start building some worldwide database of non-English blogs worth subscribing to for everyone to find something interesting in another language?

But my major concern here is quite usual: instead of actually reading anything you will see a text displayed in the language of your choice - but in many cases lacking any meaning or understandability at all. Of course the English-only blogosphere is very excited about the functionality (as just about everything Google comes up with) but I have my doubts about people speaking having languages other than English as their native ones. There are obviously some language pairs that work more or less acceptable while others lack coherence entirely.

As a native Russian speaker I started my research with the Russian language, obviously. So first of all, I tried to translate the feed with Techmeme top stories I am subscribed to into Russian. As you know if you are subscribed to Techmeme top stories yourself, this feed only contains titles of the posts or articles, names of their authors along with the publications they appear in and a short introduction from the article itself. I guess this feed must be quite a valid example since if I did not know English but wanted to be up to date with the most discussed technology events, I’d definitely choose Techmeme for my quick share of news.

I actually hoped the translation would provide a feed in Russian that will be understandable enough to get a general understanding of what is going on in the technology world but to my dismay it just did not happen and if I did not read these very news in English before, I swear I’d never been able to understand what they were about even remotely. I hope it is clear that I turned the translation setting for this feed off immediately after completing the post. And of coursed I switched my language in the account settings back to English as Google Reader itself in Russian is pretty difficult to use.

I know that my own opinion should not be trusted 100% so I went looking for additional evidence in the Russian-language blogosphere. All of the tests carried out by bloggers here in Russia had some hilarious examples of translations Google performed for their feeds - and this should have been expected. After reading a few of the Russian-language bloggers talking about the service, I have a feeling that people are not appalled by the service as it can produce decent results for some general news topics (at least to understand what countries or people the piece of news in question is about). But of course the agreement is that if you know English enough to understand general meaning of a text, you’d better off trying to read it in English than struggling to understand what the Russian translation is about.

The most important thing that I’ve seen on blogs in French and Spanish is that people admit the translation is not perfect but the translation system is getting better and I think this must be the best compliment Google could get for the efforts.

I have also checked the comments to the English-language blog posts covering the announcement from people native in other languages sharing their experience with the new feature. People reported poor translation from English to Chinese and Japanese along with some “Asian” language which I have no idea about. There were some language pairs where people reported decent translation - like English to Italian - and it is understandable as some languages are just easier handled by machine translation systems than others.

But unfortunately there are languages that are simply beyond efforts of any machine translation system I’ve ever heard of - my own native Russian including. And if you choose Google to help you read things written in these languages or to translate feeds into these languages for you, you will be sure to have lots of hilarious examples of machine translation quality and tons of time spent trying to figure out the meaning of what you are supposed to understand from the translation.

At least the team over at Google Reader promises that our reading experience will be getting better as the translation technology develops and gets better itself. But my question is why they did not wait until the automated translation technology is mature enough to provide coherent reading experience instead of just hilarious things in your feeds or at least roll the functionality out only for the languages that are reported by users to produce acceptable quality of translation.

Google Reader translated Techmeme feed into Russian

Google Search Translation Service Tested

Google Releases Online Translator Bot

Google Takes Pride in Not Paying Translators

Google Suggest Denies Support of Non-English Languages But Already Offers It

39 Comments (Subscribe to rss)

StareClips.com - November 11, 2008 at 05:33 pm PDT

It seems like this blog post was translated using Google’s automated translation software.

If you understand anything about machine translation, you would know that it needs “more input” as Johnny-5 would have said. So, if Google waited for its service to become “more mature” before releasing it to the masses, it would never BECOME mature. It’s a catch-22.

So, what they do is they release it early. This allows a lot of feedback from a lot of people. It also allows for a lot of content to be run through the system. Through this activity, it can get better. The same thing was done for GOOG-411. It was a bit early, but they released it to get a bunch of people talking into it. Eventually, they are likely going to roll the service into the GrandCentral service and combine it with Android phones.

Google’s philosophy isn’t “get it 100% before releasing it”… it’s “release early, release often.” It’s a formula that has worked for them well. Sure, there will always be those who nitpick about every little detail shortly after a new experiment is released… but a few years later, everyone talks about how Google is leaving everyone else in the dust.
Jollo Admin - November 11, 2008 at 05:56 pm PDT

Hi,

This is a very interesting article.

At Jollo (www.jollo.com) we compare machine translation services, like Google, with humans, and found that even Google’s service deteriorates for longer translations.

Jollo Admin
Durf - November 11, 2008 at 07:18 pm PDT

There are a number of translators in the English-language blog world who aren’t too impressed with Google’s (or anyone else’s) MT efforts, for what it’s worth.
Jason - November 11, 2008 at 08:10 pm PDT

I think it is good if you speak the language but just need a gist translation. Sometimes if I’m doing something like spanish and can’t think of how something should go I’ll type it in and be like Oh yea, modify it to be correct. So it has its uses however it really isn’t 100% accurate but still useful.
Ori Matalon - November 12, 2008 at 12:37 am PDT

I think you can enjoy this service, if you set your expectations right. What I mean is that it can assist you to understand in general what a page is about. Then, if you’re interested in understanding exaclty what’s there - Google is not (currentl) the solution.
Svetlana, great and interesting article!

my comments at http://www.commentino.com/orim
trevelyan - November 12, 2008 at 12:56 am PDT

I don’t know what the skeptics are talking about here. Google has fantastic machine translation, and has really pushed forward the state of the art under Franz Och. Their Chinese-English translation service is really good. It suffers from a lack of understanding of Chinese grammar, but that is a problem with SMT systems. As far as things go, they are good.

My company provides Chinese language training, and one of the things we know is that the people who use MT most frequently are often those with experience in the language who understand immediately when a machine is getting things wrong and why. MT is not only for non-speakers, although it has less and less limitations.
Svetlana Gladkova - November 12, 2008 at 01:13 am PDT

StareClips: I will ignore your idea about me translating this post using automated translation system as I don’t think you could leave a comment on a post you completely failed to understand.

And yes, I do know that machine translation systems can aggregate knowledge suggested by users to eventually improve the way they work but I have not seen any tool to report wrong translations and suggest better variants in the Google Reader. So how is it supposed to get better in this particular case? I’ve never objected release early idea when users’ input can help improve the product but this one does not look like the case.
Svetlana Gladkova - November 12, 2008 at 01:14 am PDT

JolloAdmin: Thanks for mentioning your service, I will be sure to play with it a bit myself though I know it is not fair to compare human translation with that produced by a machine.
Svetlana Gladkova - November 12, 2008 at 01:18 am PDT

Durf: That’s exactly what I was trying to say in the post: I am not criticizing Google for the efforts, I just don’t like machine translation at all and I happen not to see any sense in it, unfortunately.
Svetlana Gladkova - November 12, 2008 at 01:22 am PDT

Jason & Ori: I’m afraid that we must be talking about different language pairs with you probably because in my English & Russian test the problem was that it was actually absolutely impossible to get any meaning from the translated text - hence my disappointment. I did not expect a lot, I just expected it to bring the general meaning as other users reported it did. And I never expect machine translation to be anywhere close to 100% correct so maybe it was just my languages that were that bad in this particular case.
allen - November 12, 2008 at 04:31 am PDT

great post svetlana!
Gwhiz - November 12, 2008 at 04:31 am PDT

Seriously, if into English is one of your language pairs, you should give up and use someone’s MT. This article was so poorly written, I only skimmed to the bottom–I too assumed you had use Google to translate this to prove a point: MT is still a maturing technology and it doesn’t result in note-perfect translations. It’s sort of funny that you’re proving a different point for MT unintentionally–if a high-quality, professional human translation isn’t an option for whatever reason(s), you might as well get a low-grade MT because you’ll get the gist of it even if the grammar and style suck. Rock on!
quirkyalone - November 12, 2008 at 04:42 am PDT

Hi,
Maybe this must be a problem of the Russian-English pair specifically. I tried French->English and German->English translation before, and the general meaning is totally understandable.

Maybe the Russian language is too different from English, or (more probably), they do not have enough input to train their algorithms. But overall, I think they are heading in the right direction with the automatic translation, but of course there is still a lot of work to be done.
Svetlana Gladkova - November 12, 2008 at 04:47 am PDT

Thank you Allen, glad to see you stopping by here
Andy Beard - November 12, 2008 at 04:48 am PDT

I have ben heavily involved with computer assisted translation in the past when I was at Techland, and a while back was highly critical of the crowd-sourced translation of Facebook creating embarrasing translation that at times was worse than Google could achieve automatically.

I think you might need to switch to a non-translator brain to really understand the benefit.

My primary role at Techland was all their international sales and purchasing, but that also meant I in some ways managed the localization process either too or from Polish into primarily English.

Despite living here now almost 15 years, I don’t regard myself as fluent in Polish - in fact I don’t even try hard to improve niw, and am more concerned about a degradation in my written and spoken English.

Whilst I could hardly speak or write Polish

1. I translated manuals and help texts for 3 products Polish > English
2. I corrected the translation from professional translators for in game text, audio, manuals, boxwork, marketing materials on 3 products

You might find this one strange

3. I corrected the English > Polish translation on over 100 SKUs of localised manuals and software. The translation had been performed by highly qualified translators, but sometimes they totally *&%^&@ up the meaning or the translation was too stilted.

That was all achieved using a computer aided translation product we developed, which itself was highly critcised by some in the media, and many translation professionals.

I have managed to use poorer quality translation products (imho) such as babelfish to visit a Russian blog that linked to me, understand their complete post which was quite long, understand 80% of the comments, and make relevant comment in English.

I am a strong believer that a subject matter expert can use machine translation to gain a strong overview of the content of any document written about the subject matter they are an expert in, in most languages.
Svetlana Gladkova - November 12, 2008 at 04:49 am PDT

Gwhiz: I will be sure to ask all the 60k subscribers to immediately unsubscribe from this blog to avoid my poor English, appreciate your recommendation.
Svetlana Gladkova - November 12, 2008 at 05:04 am PDT

@quirkyalone: Yes, that could very much be possible. I remember Google even admitting Russian was one of the most difficult languages for them somewhere on the official blog. And this is exactly why I think it should not be necessary to launch this functionality for all the languages - they could choose to only offer those that have positive feedback from users and keep Russian (and other languages in the same category) in the training mode. But of course it is understandable that Google always needs everything to be global from the very beginning.
Svetlana Gladkova - November 12, 2008 at 05:13 am PDT

Andry, thanks a lot for the insightful comment and the description of your experience with Polish and automated translation. I believe you are correct thinking that in some sense I still think as a translator myself and this issue is just very sensitive to me because of this.
I agree with you 100% that in some cases crowdsourcing can produce even worse results than Google’s MT (Facebook is terrible, Google is poor, Plurk is hilarious - all in Russian, I mean) but to me the real question is why it is Google of all companies trying to prove the world will be better off without language professionals sticking to Google’s translation tools. I don’t understand why Google chooses not to pay to translators for localization of their own products as I do believe they can afford it perfectly well and I don’t understand why they need to push machine translation everywhere.
There are situations where it can be useful (like your examples) but reading prose delivered via RSS does not seem like one of that situations - to me at least. I can be mistaken and others could really understand the general meaning of the written texts delivered in this manner and get enough information from it but unfortunately my testing did not provide anything that could even remotely resemble the meaning of the texts - hence this post.
StareClips.com - November 12, 2008 at 07:43 am PDT

You say that I completely (as in 100%) failed to understand your article. I don’t believe it is that black and white, which suggests to me that you believe the world is this way.

From what I understand, you feel that if something is going to be translated, it better be translated perfectly, or else there is no point in translating it. Again, this is another very black & white argument.

I think the point YOU are failing to understand is Google is not trying to translate the Bible… or poetry… or great works of literature. They’re trying to translate blog posts. In general, these are short snippets of information. Chances are, the original post was written poorly as it is. So, if Google’s machine translation service mangles it a bit, it shouldn’t matter.

Again, what Google is doing is trying to make MORE information more ACCESSIBLE to people. Here is an example:

Let’s say I am using the Google Chrome browser. And let’s say I get an error message that says “error 4″. Let’s say I search the Internet and I cannot find anyone who got the same error and who was able to solve the problem. Now, let’s say someone in Canada who posts a blog in French posts about getting “error 4″ in Google Chrome, but they solved the problem. If solving the problem involved uninstalling Chrome, uninstalling Flash, re-installing Chrome, and re-installing Flash… don’t you think this general concept might still come across, even if the grammar is mangled by Google’s machine translation software?

The point Google is trying to make with their automated machine language translation software is to make certain bits of information more accessible to people who may need that information… not in trying to translate great works of literature and keep it true to its meaning.

As for why Google uses crowds for translation services… this is because Google only makes money on one thing. Advertising. Everything else is more or less an experiment in bringing more traffic. So, why should they spend a bunch of money having professional translators making an experimental service available in 40+ languages only to have this experiment trashed a couple of years later because it just didn’t pull in the necessary numbers?

Instead, they use crowds to loosely translate products to bring in traffic from as many sources as possible, without spending a bunch of money. If the service really takes off and is successful, they hire professional translators to fix the bad translations. This is a whole lot more logical. They aren’t trying to replace the translators of the world… they’re trying to make translation more accessible. If my options as an individual are to pay more than $20 for a translation, or to get a poor translation for FREE, I’m going to go the FREE route… because I’m not making money from my websites… they are just a hobby.
Brian - November 13, 2008 at 06:56 pm PDT

I don’t think anyone can argue with the fact that professional translation is undoubtedly better than MT right now, but I don’t understand your complaint: “why it is Google of all companies trying to prove the world will be better off without language professionals sticking to Google’s translation tools.”

Obviously, all content creators that publish RSS feeds can’t afford to pay for professional translation services - isn’t some translation (even if obviously isn’t perfect) better than nothing?

I use the feature to get a decent understanding of a lot of foreign Twitter posts that I would otherwise have no hope of understanding, and it at least seems useful to me in that regard.
Svetlana Gladkova - November 14, 2008 at 04:47 am PDT

StareClips: When two people need to understand each other they will need to really want to understand each other and they will even if they speak different languages. Unfortunately this is not our case 

So to address your points:

No, I don’t think everything should be translated perfectly or should not be translated at all. I’ve seen enough translations in my life that were far from perfect but helped people anyway. I just feel that translation quality should meet the purpose of the translation process. And unfortunately I feel that translating prose (and usually we subscribe to blog posts in RSS feeds) needs to be handled better than any machine translation can ever manage to do.

Again, as some comments here suggest, Google must perform better for some languages and worse for others. Russian seems to be among those where I could not see any general meaning from the translation at all – hence my disappointment. If you are lucky with the language pairs you tested with, good for you and I’m glad it works.

As for your example about Chrome blog post explaining how you could find a solution to a problem of the browser crashing in a French-language blog post, I think you are quite right BUT I would not expect you to sit and wait for your Google Reader to deliver you that specific blog post in French and will choose to do a Google Search which has had translation functionality for a while. So why translating RSS at all? Besides, there you also have functionality to report mistakes and suggest better translations which actually helps improve the system as well. So I think machine translation can make perfect sense there but does not seem to be particularly needed here.

And I don’t agree with your crowdsourcing arguments at all. Google is certainly not the company that can not afford professional translation services – even if it won’t make a dime on the other products like it does on advertising. And my biggest concern is that Google introduced croudsourcing translation as something of a trend online and we see multiple startups and even established companies going the same way – offering users terribly translated pages as a result. Believe me, I have often tried to use a localized version of various services but I always stick to English versions because too many of them are translated by people who have no idea of what professional translation is at all. It’s just not good for a company with Google’s potential to serve as an example for others in this particular field – unless Google really hates everyone in the world who does not speak English and wants us to be constantly irritated trying to use localized web services.
Svetlana Gladkova - November 14, 2008 at 04:53 am PDT

Brian: This particular point is not about RSS translation, it is about Google’s overall desire to stick to machine translation everywhere - look at the links to my other posts in the beginning of this one and you will understand my desperation trying to express why I think Google should not do this to translation industry.

As for web publishers unable to afford translators to provide content in other languages, this is usually the issue of value - if such a service will be needed enough for the audience reading the publication (and by needed I mean they will visit often enough and watch enough ads), the publication will be sure to launch local versions in other languages - we see it with many news outlets. But the thing is that the majority of web publishers does not even need to deliver multi-lingual content to readers as no one will need it.

And again, my post is due to the fact that I can not describe my experience with the words “far from perfect” everyone wants to use in this case. My experience was obviously terrible.
french translation - November 17, 2008 at 04:41 am PDT

Well, Google is trying to tap a wide audience by introducing translation of pages, RSS, etc. I think as time goes by there will be less and less work for translators, especially once google releases their Translation Center.
Svetlana Gladkova - November 17, 2008 at 10:14 am PDT

Yes, the day Google releases their Translation Center is certainly the day many language professionals dread even now but I do hope that there will still be companies that will choose quality for a reasonable price to cheap yet poor translations - and this is exactly what I expect them to produce. Too bad Google does not value language professionals but I am kind of sure reasonable companies will stay reasonable with their translation workflow anyway.
Bob Moniker - November 25, 2008 at 11:35 am PDT

Seriously, for a translation professional, your prose is pretty horrible. Unless you’re going for the “talks fast and joins sentences with ‘like’ instead of inhaling” sort of voice… Way too many subject shifts to make for comfortable reading.
Svetlana Gladkova - November 26, 2008 at 01:23 am PDT

Bob, sorry to disappoint you but my prose is pretty comfortable for the 60k of my readers here so I think I’ll have to ignore your comment - even though I appreciate the criticism as always.
pamela - May 20, 2009 at 11:47 pm PDT

it's interesting what you're saying…
Tatil - May 24, 2009 at 01:01 pm PDT

I think this article has been translated into other languages in the converter to use a car very enjoyable
Diet Capsules - September 13, 2009 at 05:06 am PDT

its cool site, its cool work!!!
Kfc Online Menu - September 14, 2009 at 02:03 am PDT

good site, good work!!!
young erotic - September 28, 2009 at 03:51 pm PDT

I think its very nice!!!
Tamiflu swine - October 01, 2009 at 08:28 am PDT

system divisions ngos disaster ordinances tippers deputy expressing warranties slow modes consumers
Ativan no prescriptions - October 05, 2009 at 02:12 am PDT

transport erased aspx yrfashion bound dunleavy surgeons mnns roche mlanet alliteration recoup
Valium Buy Online - October 05, 2009 at 02:12 am PDT

forsk institutes gollum stipendwhen perceptive recognized pharmatrak confirm graphics importation backdrop saracen
Valium Buy er - October 06, 2009 at 02:04 am PDT

frequent bharuch often succeeded troughs zgptpielwp affiliate origin louise estoniatel parking recurrent
Ambien no prescriptions b - October 06, 2009 at 07:20 am PDT

greatly aacsb five hadnt cataracts counterparts ranges athletes eftlvzsv stringless excavation wisconsin
Valium Buy cheap - October 07, 2009 at 06:39 am PDT

cartago iicosmetics scrambling curvy warsaw stretches plastics ncdot immunology images complexity roundabout
Termal Oteller - October 31, 2009 at 05:20 am PDT

Great post and idea Daniel! May I translate your quotes compilation into my native language, Bahasa Indonesia? Of course I will make credit and link to this post. Hope you don’t mind.
gezi rehberi - October 31, 2009 at 01:10 pm PDT

hope and pray that viewers are savvier than we think. I don’t find commercials very interesting or entertaining usually, but if they are, I will watch them online. They seldom contribute to me buying something I don’t want or need.

Leave a comment (We support avatars from Gravatar, MyBlogLog, and FriendFeed)