New Beta 3.2.98

Hello, since the new update to the OCR, the Screen OCR seems to be giving me this message no matter what. Before I could just drag the little window over whatever I wanted the OCR to parse. Note how actual characters are blocked out, but clickable as with 进士 in my screenshot.

Am I misunderstanding something about the new update or is this a bug of some kind? I'm on Google Pixel 7. Hope I'm not repeating anything, as far as I could tell the already discussed issues were different. Screen Reader seems to work as always, if not better.

Thanks. Does the message go away if you disable the option to hide recognized text? (We actually were planning to not offer that option with the new engine, since unfortunately while it’s generally more accurate it’s less precise about reporting character positions)

mikelove said:

Thanks. Does the message go away if you disable the option to hide recognized text? (We actually were planning to not offer that option with the new engine, since unfortunately while it’s generally more accurate it’s less precise about reporting character positions)

Click to expand...

Yup, that fixed it, thanks!
Another thing I forgot to mention: when I enable the legacy OCR algorithm option, the green frame/window pops up for a second, only to then crash. As of right now the legacy option seems unusable for me.

I must admit that I thought the legacy version of the screen reader to be more suitable for my needs (but I recognize that I haven't had much time to fully test out the new version of course, so not a definitive statement!). Precision in reporting character positions does seem to get lost as you've mentioned, before I could even parse a whole manga page and it would recognize everything with the Google OCR most times. Something that doesn't work on my iPad or on the new algorithm it seems.

Thank you for your continued hard work on this great app!

jirou said:

Another thing I forgot to mention: when I enable the legacy OCR algorithm option, the green frame/window pops up for a second, only to then crash. As of right now the legacy option seems unusable for me.

Click to expand...

Thanks, just uploaded a fix for this.

jirou said:

Precision in reporting character positions does seem to get lost as you've mentioned

Click to expand...

Yeah, not much we can do about that one unless we want to never update our OCR algorithm, and for anything other than clean straight on-screen text like with Screen OCR the need for a change was pretty clear. We are looking at some other options for Screen OCR specifically, though.

jirou said:

Something that doesn't work on my iPad or on the new algorithm it seems.

Click to expand...

Sorry, could you go into a little more detail about how the behavior on the iPad differs from that in old Android system? iPad uses the exact same algorithm, so if it's not working on there then that suggests something else wrong besides the algorithm change.

Possibly found a bug that hasn't been in version 97. Sometimes the previous selected main page (history, continue test, organize) will stay blank after jumping to another word in the opened entry, mainly after searching. I often encountered that while doing searches in dictionary during a test.

I can reproduce it with the history screen by exactly following these steps: fresh open pleco, select history, select dictionary, search for any entry you like, open it. Then in the headword or description, click on another 汉字 (one without a direct link) to open it's description popup, then click the search glass icon to search for that entry, finally open that entry from the search result list. Then use the back icon and open the history screen again, it just shows "Pleco" as headline.

First found that in build April 2nd, currently recorded in the 8th of April version.

Thanks - reproduced and fixed for the next build.

The next beta - now in review - adds a new option in Settings / OCR for *another* new OCR algorithm, this one an open-source algorithm from Baidu, just for still + screen OCR. They have a whole bunch of them and we picked the biggest / slowest one to see if it provides any meaningful accuracy improvements over Google's algorithm in exchange for that extra size and performance cost. (our testing suggests it might)

mikelove said:

Sorry, could you go into a little more detail about how the behavior on the iPad differs from that in old Android system? iPad uses the exact same algorithm, so if it's not working on there then that suggests something else wrong besides the algorithm change.

Click to expand...

What I meant was that, on my Google Pixel, the Still OCR (as well as Screen OCR) had previously allowed me to parse the whole screen very accurately even if there were images involved, like in a manga for example.

On the iPad on the other hand the OCR works beautifully for regular PDFs that are just text, but barely recognizes anything when scanning a whole manga page at once. I know there are options between using Apple's OCR, Pleco's OCR, and a combined method but in the case of a manga page it doesn't seem to matter and none of them really work. Tried on Legacy Pleco and the Beta.

I know this use case probably isn't the priority lol, it was just strange to me given Google's OCR seemed to work flawlessly vs. Apple's. Though I have to add the iPad I use is old - a 2017 iPad Pro. It's on iPad OS 17.7.5, perhaps the hardware is just to old to make use of the latest versions of Apple's OCR?

mikelove said:

Yeah, not much we can do about that one unless we want to never update our OCR algorithm, and for anything other than clean straight on-screen text like with Screen OCR the need for a change was pretty clear. We are looking at some other options for Screen OCR specifically, though.

Click to expand...

Completely understandable!

mikelove said:

The next beta - now in review - adds a new option in Settings / OCR for *another* new OCR algorithm, this one an open-source algorithm from Baidu, just for still + screen OCR. They have a whole bunch of them and we picked the biggest / slowest one to see if it provides any meaningful accuracy improvements over Google's algorithm in exchange for that extra size and performance cost. (our testing suggests it might)

Click to expand...

Excited to try this one out as well.

Thanks again for your continued development, fast responses and open communication!

jirou said:

I know this use case probably isn't the priority lol, it was just strange to me given Google's OCR seemed to work flawlessly vs. Apple's. Though I have to add the iPad I use is old - a 2017 iPad Pro. It's on iPad OS 17.7.5, perhaps the hardware is just to old to make use of the latest versions of Apple's OCR?

Click to expand...

What's confusing is that the old algorithm is literally identical - same exact ML model - between iOS and Android. It's ancient - originally developed for scanning business cards on Windows Mobile devices - so there should be no performance issue on a 2017 iPad, and no difference in general between how it performs on Android and iOS.

Perhaps it has something to do with the way the PDF is being rendered, resolution or some such - I don't suppose you have an example of a file that works well on Android and badly on iPad that you could email or PM me?

Both of the new algorithms are also available on iOS, so while we're currently feeling like Apple's algorithm (which we're likewise making the primary / default option in the next 4.0 beta, even for live OCR) is good enough - and has the advantage of being built into the OS so it's zero maintenance and doesn't take up any space - if people find that one of the Android ones is better for some use cases it wouldn't be hard to support it too.

Great to see the new features listed and look forward to trying to them out! Just came across a bug using the handwriting recognition to look up the character 䞍. The new handwriting algorithm absolutely refused to recognize it (as did handwriting input in Gboard—is there a connection?), but after some frustration and using an online character database to confirm the character and confirming it was in Pleco, I then turned on "Legacy HWR algorithm" and Pleco immediately recognized it. On a sidenote, I also couldn't find it through radical input (before I moved onto the other steps).

Thanks.

䞍 is in the rare character block CJK Unified Extension A, so I'm afraid this is actually the expected behavior with the new HWR - support for ExtA was the main feature of our 'enhanced handwriting' add-on, and after virtually nobody bought that we concluded it wasn't something people cared about enough to keep offering it, when doing so a) entailed an expensive fixed annual license fee which those sales weren't remotely covering (and which we could not plausibly imagine was convincing many people to buy bundles) and b) introduced an extra 6,582 potential false positives, since people who got that add-on in bundles rarely knew about the option to disable recognition of rare characters.

As a practical matter, the main purpose of offering handwriting input in Pleco nowadays - despite the availability of free Chinese handwriting keyboards built into iOS and Android - is "make it really easy for people to input characters with handwriting without having to figure out a bunch of settings screens," so that's kind of what we have to optimize around; offer something similar to those keyboards in accuracy, but which you can get at on any device with a single tap without having to figure out how to turn on Chinese input support on your phone. (back in 2009 our handwriting was also a lot more accurate than theirs, but it's hard to make that argument now)

But we have no plans to drop the 'legacy algorithm' option for people who want that extra character set support - we never want to take away anything people already had, we have dictionaries in our catalog we've stopped selling many years ago but continue supporting (and even updating to new data formats). We just didn't see enough benefit in it to continue selling handwriting with ExtA support to new customers.

Going forward, we'd eventually like to offer our own HWR models with even greater character set support, and if there's a lot of demand then we're happy to look into licensing a model with expanded support where the price is more reasonable or where we can pay per-copy royalties, but the nearer-term alternative is that 4.0 is shipping with a multi-component search feature as an alternative to radical input and that has much greater character set support (we've only deployed extension A in the 4.0 betas so far, but the data we have goes all the way through extension F - almost 89,000 characters).