Integrating translation into Google Goggles

Yesterday, at the Mobile World Congress in Barcelona, Eric Schmidt and I demonstrated a prototype version of Google Goggles that showcases the potential of integrating Google's machine translation and image recognition technologies. In the video below, you'll see how we use Goggles to take a picture of a German menu and instantly translate the text into English:

You may wonder what's happening in the background. On the simplest level, this prototype connects the phone's camera to an optical character recognition (OCR) engine, recognizes the image as text and then translates that text into English with Google Translate.

Right now this technology only works for German-to-English translations and it's not yet ready for prime time. However, it shows a lot of promise for what the future might hold. Soon your phone will be able to translate signs, posters and other foreign text instantly into your language. Eventually, we're hoping to build a version of Google Goggles that can translate between all of the 52 languages currently supported by Google Translate — bringing even more information to you on the go.