Screen OCR + Screen Reader

[mikelove]

No, for a number of reasons:

1) Some characters which are used by themselves have multiple traditional mappings depending on meaning; 里 for example stays 里 for "mile" but becomes 理 for "inside."

2) Some multi-character words also do this, 面 in 冷面 for example stays 面 when referring to someone's face ("cold face" - poker face) but becomes 麵 when referring to noodles ("cold noodles").

3) No dictionary covers 100% of all Chinese words, particularly not when you factor in proper nouns.

4) Even with a perfect word database, breaking up Chinese text into individual words ("word segmentation") is hard because there are no spaces - you often run into a situation where a string of characters could be broken up into words in multiple different ways. Even worse when you factor in words that can be split in two. You can get reasonably good results (as we do now) by trying to find the breakdown that results in the highest-frequency / most common words, but it's far from 100%.

5) There isn't even a consistent definition of what constitutes a 'word' in Chinese - the boundary between 'word' and 'phrase' is somewhat nebulous and different groups have different standards for where to draw the line.

 

[sangormam]

Thanks for this explanation

No, for a number of reasons:

1) Some characters which are used by themselves have multiple traditional mappings depending on meaning; 里 for example stays 里 for "mile" but becomes 理 for "inside."

2) Some multi-character words also do this, 面 in 冷面 for example stays 面 when referring to someone's face ("cold face" - poker face) but becomes 麵 when referring to noodles ("cold noodles").

3) No dictionary covers 100% of all Chinese words, particularly not when you factor in proper nouns.

4) Even with a perfect word database, breaking up Chinese text into individual words ("word segmentation") is hard because there are no spaces - you often run into a situation where a string of characters could be broken up into words in multiple different ways. Even worse when you factor in words that can be split in two. You can get reasonably good results (as we do now) by trying to find the breakdown that results in the highest-frequency / most common words, but it's far from 100%.

5) There isn't even a consistent definition of what constitutes a 'word' in Chinese - the boundary between 'word' and 'phrase' is somewhat nebulous and different groups have different standards for where to draw the line.

 

[sangormam]

1.When i have enabled screen reader ocr on galaxy s5 and when after some time of inactivity my phone locks,
its hang - i have to remove battery or press power multiple times and move finger on black screen to unlock it.

2.Can you add option to move green ocr box without re-sizing it?

 

[mikelove]

1) Is this while you're actually in the middle of OCR (so the characters are showing right then) or just while it's active in the background?

2) Possibly - we'd actually been planning to do that via Force Touch on iOS but we can probably come up with a sensible way to do it on Android too.

 

[sangormam]

1. I think it happened in both cases.

 

[etm001]

There's also an option to trigger it with a notification now, or a shortcut intent.

Can you clarify what "trigger with a notification" means? I see how I can send something directly to Pleco via a share menu, but I'm not sure what triggering the screen reader with a notification means.

Also, the screen reader functionality is really cool. I was missing the web browser mode available in iOS, but the screen reader pretty much makes up for it.

 

[mikelove]

@sangorman - sorry I didn't reply sooner; we'll investigate this and hopefully have a solution soon.

@etm001 - there's an option in Settings / External Access to use a notification instead of a floating button to activate Screen Reader; basically you just swipe down the notification shade at the top of the screen and then tap on a button.

We still might bring Web Reader to Android eventually, but we're in much less of a hurry about doing so thanks to Screen Reader.

 

[Fox]

Sometimes when I use the screen OCR, it inexplicably fails to even recognize large blocks of computer-text Chinese characters. For example, while using WeChat, sometimes it will recognize all the text in the area enclosed by the green block, but sometimes it will simply ignore a small portion of it, or even the vast majority of it. Resizing the block and restarting the screen OCR does not seem to change it; it seems to be very systematic in terms of what it recognizes and what it does not, but the underlying principle isn't clear. Sometimes changing the size of the text itself affects this, but sometimes it does not as well.

 

[mikelove]

We're working on that one, both with better text detection and a color-sensitive upgrade to our OCR algorithm. In WeChat though you're probably better off with Reader than OCR - we've specifically optimized it around a few common apps like WeChat.

 

[Moro]

I'd like to second the request for the ability to move the green box. Maybe an option that specifies what happens when you grab a side of that rectangle, toggle between resizing and moving?

In fact what I find even more necessary is the ability to start the green box anew instead of scrolling back to where it was and dragging it to a new place. My use case for all of these is when you are reading something that has a mixture of text and pictures while zoomed in. Text with diagrams maybe, or a manga/manhua or any text where the small screen or tiny fonts make it necessary to zoom in and scroll around. It would be great to scroll to a new place in the picture/file and to be able to just make a long press and drag a new green OCR box around the interesting text.

This is relevant to all of OCR, not just Screen OCR, but I didn't find a forum topic discussing it.

 

[mikelove]

The hope is that better text detection will make this less necessary - i.e., with a still image you generally wouldn't even need the box because Pleco will accurately pick out + OCR the text regions from the image, even in a tricky case like a comic book. But more flexible invocation of the green box makes a lot of sense too.

 

[Moro]

The hope is that better text detection will make this less necessary

Well, of course I hope that we can have both Better text detection and a "long-press to draw a new green box".

 

[Kent Wheeler]

I started using this function yesterday. But today my green box does not appear. Everything else functions the same.

 

[Kent Wheeler]

I found the problem, the green box was last used with the screen portrait and the box was at the bottom of the screen. Now when I use tablet in landscape the green box does not appear because it must be plotting off the page.

I have changed the view to portrait, moved the box back to the top of the screen and then changed back to landscape. the box now appears as I expect it to.

 

[mikelove]

Great! We'll add a fix for that the next time we update our Android app.

 

[Kris]

Can you give us a time table for the iOS version? Will it work with force touch only (newly Iphone SE owner here) and how will it be integrated on iOS in general? I loved the screen writer while still having Android. The communication with my chinese family in law instantly became much more sophisticated.

my face when I found out about screen writer

 

[mikelove]

It'll be in our next major update but we don't know when that'll be out. Won't be quite as convenient as the Android one in any case, unfortunately - would be an App Extension so it would take a few extra taps to get to.

 

[sangormam]

@mikelove
i just downloaded new update on ipad and one nice feature - constant clipboard monitoring for splitview mode compatibility could be also implemented on android?
Cause i am using multitasking tools on samsung galaxy phone - there is multi window mode so i can have few apps simultaneously open on one screen just like in ios.

 

[mikelove]

We actually do that already, and automatically / by default since we can do it on Android without wasting much power or CPU; is it not working on your phone? Could you let me know the model + firmware version?

 

[sangormam]

We actually do that already, and automatically / by default since we can do it on Android without wasting much power or CPU; is it not working on your phone? Could you let me know the model + firmware version?

Sorry i just assumed that it doesnt work cause i mostly use ios.
But sometimes android.
and on ios i saw this new update; on android i didnt see anything new in play store for pleco.
So i assumed that it is only in newest update for ios.

It works fine i checked it and everything is great.
thanks.