This is pretty cool, actually - the framerates would be a little dicey if we tried it now, though, recognizing alphabetic languages is a lot easier than recognizing Chinese. (framerates are why we currently only recognize characters inside of the magic green box) The translation part would be a little challenging too - even if we could come up with a decent real-time translator, the output it generated would be a whole lot wordier than the Chinese and you'd be left with really tiny English translations that barely fit on the screen.
But in theory, we know what the characters are, we know where they are in the image and we can easily detect what colors they're being drawn in, so this wouldn't even be that difficult if we could get the performance and translation quality up to snuff.
Definitely makes me feel good about not wasting time adding OCR support for English, though.