New Beta 3.2.98

[jirou]

Hello, since the new update to the OCR, the Screen OCR seems to be giving me this message no matter what. Before I could just drag the little window over whatever I wanted the OCR to parse. Note how actual characters are blocked out, but clickable as with 进士 in my screenshot.

Am I misunderstanding something about the new update or is this a bug of some kind? I'm on Google Pixel 7. Hope I'm not repeating anything, as far as I could tell the already discussed issues were different. Screen Reader seems to work as always, if not better.

 

[mikelove]

Thanks. Does the message go away if you disable the option to hide recognized text? (We actually were planning to not offer that option with the new engine, since unfortunately while it’s generally more accurate it’s less precise about reporting character positions)

 

[jirou]

Thanks. Does the message go away if you disable the option to hide recognized text? (We actually were planning to not offer that option with the new engine, since unfortunately while it’s generally more accurate it’s less precise about reporting character positions)

Yup, that fixed it, thanks!
Another thing I forgot to mention: when I enable the legacy OCR algorithm option, the green frame/window pops up for a second, only to then crash. As of right now the legacy option seems unusable for me.

I must admit that I thought the legacy version of the screen reader to be more suitable for my needs (but I recognize that I haven't had much time to fully test out the new version of course, so not a definitive statement!). Precision in reporting character positions does seem to get lost as you've mentioned, before I could even parse a whole manga page and it would recognize everything with the Google OCR most times. Something that doesn't work on my iPad or on the new algorithm it seems.

Thank you for your continued hard work on this great app!

 

[mikelove]

Another thing I forgot to mention: when I enable the legacy OCR algorithm option, the green frame/window pops up for a second, only to then crash. As of right now the legacy option seems unusable for me.

Thanks, just uploaded a fix for this.

Precision in reporting character positions does seem to get lost as you've mentioned

Yeah, not much we can do about that one unless we want to never update our OCR algorithm, and for anything other than clean straight on-screen text like with Screen OCR the need for a change was pretty clear. We are looking at some other options for Screen OCR specifically, though.

Something that doesn't work on my iPad or on the new algorithm it seems.

Sorry, could you go into a little more detail about how the behavior on the iPad differs from that in old Android system? iPad uses the exact same algorithm, so if it's not working on there then that suggests something else wrong besides the algorithm change.

 

[Dkhr77]

Possibly found a bug that hasn't been in version 97. Sometimes the previous selected main page (history, continue test, organize) will stay blank after jumping to another word in the opened entry, mainly after searching. I often encountered that while doing searches in dictionary during a test.

I can reproduce it with the history screen by exactly following these steps: fresh open pleco, select history, select dictionary, search for any entry you like, open it. Then in the headword or description, click on another 汉字 (one without a direct link) to open it's description popup, then click the search glass icon to search for that entry, finally open that entry from the search result list. Then use the back icon and open the history screen again, it just shows "Pleco" as headline.

First found that in build April 2nd, currently recorded in the 8th of April version.

 

[mikelove]

Thanks - reproduced and fixed for the next build.

 

[mikelove]

The next beta - now in review - adds a new option in Settings / OCR for *another* new OCR algorithm, this one an open-source algorithm from Baidu, just for still + screen OCR. They have a whole bunch of them and we picked the biggest / slowest one to see if it provides any meaningful accuracy improvements over Google's algorithm in exchange for that extra size and performance cost. (our testing suggests it might)

 

[jirou]

Sorry, could you go into a little more detail about how the behavior on the iPad differs from that in old Android system? iPad uses the exact same algorithm, so if it's not working on there then that suggests something else wrong besides the algorithm change.

What I meant was that, on my Google Pixel, the Still OCR (as well as Screen OCR) had previously allowed me to parse the whole screen very accurately even if there were images involved, like in a manga for example.

On the iPad on the other hand the OCR works beautifully for regular PDFs that are just text, but barely recognizes anything when scanning a whole manga page at once. I know there are options between using Apple's OCR, Pleco's OCR, and a combined method but in the case of a manga page it doesn't seem to matter and none of them really work. Tried on Legacy Pleco and the Beta.

I know this use case probably isn't the priority lol, it was just strange to me given Google's OCR seemed to work flawlessly vs. Apple's. Though I have to add the iPad I use is old - a 2017 iPad Pro. It's on iPad OS 17.7.5, perhaps the hardware is just to old to make use of the latest versions of Apple's OCR?

Yeah, not much we can do about that one unless we want to never update our OCR algorithm, and for anything other than clean straight on-screen text like with Screen OCR the need for a change was pretty clear. We are looking at some other options for Screen OCR specifically, though.

Completely understandable!

The next beta - now in review - adds a new option in Settings / OCR for *another* new OCR algorithm, this one an open-source algorithm from Baidu, just for still + screen OCR. They have a whole bunch of them and we picked the biggest / slowest one to see if it provides any meaningful accuracy improvements over Google's algorithm in exchange for that extra size and performance cost. (our testing suggests it might)

Excited to try this one out as well.

Thanks again for your continued development, fast responses and open communication!

 

[mikelove]

I know this use case probably isn't the priority lol, it was just strange to me given Google's OCR seemed to work flawlessly vs. Apple's. Though I have to add the iPad I use is old - a 2017 iPad Pro. It's on iPad OS 17.7.5, perhaps the hardware is just to old to make use of the latest versions of Apple's OCR?

What's confusing is that the old algorithm is literally identical - same exact ML model - between iOS and Android. It's ancient - originally developed for scanning business cards on Windows Mobile devices - so there should be no performance issue on a 2017 iPad, and no difference in general between how it performs on Android and iOS.

Perhaps it has something to do with the way the PDF is being rendered, resolution or some such - I don't suppose you have an example of a file that works well on Android and badly on iPad that you could email or PM me?

Both of the new algorithms are also available on iOS, so while we're currently feeling like Apple's algorithm (which we're likewise making the primary / default option in the next 4.0 beta, even for live OCR) is good enough - and has the advantage of being built into the OS so it's zero maintenance and doesn't take up any space - if people find that one of the Android ones is better for some use cases it wouldn't be hard to support it too.