Mapping my Understanding of SRS to Pleco Flashcard Settings

[gato]

There are actually 4 Card selection systems in Pleco, and Repetition-spaced is one of them - it is not a Scoring system.

Although Pleco breaks the flashcard system into a selection component and a scoring component, in reality, they work together. Spaced Repetition requires a scoring system that's customized for Spaced Repetition, which is what the "automatic" scoring sytem in Pleco is meant to be.

So one can think of the "automatic" scoring system as the "spaced repetition" scoring system under Pleco.

See this from the Pleco manual:

http://www.pleco.com/ipmanual/flash.html#scoringsettings
Automatic - flashcard scores are automatically managed by Pleco, factoring in your past history with the card (how often you've remembered it correctly/incorrectly) in order to determine how much to increase/decrease the score by after each answer; this is similar to the approach used by other systems like SuperMemo/Mnemosyne/Anki.

 

[HW60]

Although Pleco breaks the flashcard system into a selection component and a scoring component, in reality, they work together. Spaced Repetition requires a scoring system that's customized for Spaced Repetition, which is what the "automatic" scoring sytem in Pleco is meant to be.

So one can think of the "automatic" scoring system as the "spaced repetition" scoring system under Pleco.

See this from the Pleco manual:

But if you use "your" Manual Scoring System, the Scores develop the way you are accustomed to. If you then use Manual together with Repetition-spaced, the cards are selected according to their score and duration since last review. If you add a score filter - as you do in your manual scoring system - you should be very close to your fixed system - the only (?) difference being that cards are selected randomly when they are due, and fixed in "your" system.

 

[HW60]

But if you use "your" Manual Scoring System, the Scores develop the way you are accustomed to. If you then use Manual together with Repetition-spaced, the cards are selected according to their score and duration since last review. If you add a score filter - as you do in your manual scoring system - you should be very close to your fixed system - the only (?) difference being that cards are selected randomly when they are due, and fixed in "your" system.

Coming back from holidays without Pleco I found about 2000 cards to review. Therefore I decided to skip my old system definitely and to use the system described above: repetition-spaced card selection and a score filter from 0 to 100 first for the first review. This is very close to Gato's system, and after one week with it I really appreciate it: I can concentrate on cards that need a review, and I gained plenty of time to learn and repeat the language with books, mp3 etc without the dayly pressure to review. And I can pause some days without any problem.

Increasing the upper limit of the score filter is a bit tricky, therefore I would like to repeat a proposal I made some time ago: I would prefer to enter the number of cards I want to review, and let Pleco roughly estimate the upper limit of the score filter which results in a number of cards close to my wish. This seems to be easy to implement, because Pleco only has to do some iterations, trying some upper limits and comparing the resulting number of cards with the desired number. But I darkly remember that you said this iteration would not be as simple as it looks at first sight.

 

[mikelove]

Increasing the upper limit of the score filter is a bit tricky, therefore I would like to repeat a proposal I made some time ago: I would prefer to enter the number of cards I want to review, and let Pleco roughly estimate the upper limit of the score filter which results in a number of cards close to my wish. This seems to be easy to implement, because Pleco only has to do some iterations, trying some upper limits and comparing the resulting number of cards with the desired number. But I darkly remember that you said this iteration would not be as simple as it looks at first sight.

That by itself sounds very easy, but it runs into the same problem I have with limiting the number of cards in SRS in general - if done carelessly you can end up not reviewing enough and losing a lot of what you've learned. Hence my desire to come up with something more intelligent.

 

[HW60]

That by itself sounds very easy, but it runs into the same problem I have with limiting the number of cards in SRS in general - if done carelessly you can end up not reviewing enough and losing a lot of what you've learned. Hence my desire to come up with something more intelligent.

I think it is very kind of you that you feel responsible for the success of the users of the flashcard system, but actually without any change of the system I am not reviewing enough and lose a lot of what I have learned. That is due to the amount of time I can spend reviewing, and that was probably the reason why Gato invented his score filter system for busy users which I am using now. I think this system is already rather intelligent and would be even more intelligent (and easy to implement), if I could chose the amount of time = the number of cards to review.

 

[mikelove]

I think it is very kind of you that you feel responsible for the success of the users of the flashcard system, but actually without any change of the system I am not reviewing enough and lose a lot of what I have learned. That is due to the amount of time I can spend reviewing, and that was probably the reason why Gato invented his score filter system for busy users which I am using now. I think this system is already rather intelligent and would be even more intelligent (and easy to implement), if I could chose the amount of time = the number of cards to review.

My problem is that it's already much too easy for someone to configure Pleco flashcards in such a way as to prevent them from successfully learning anything, and I'd rather not add any more options that are likely to do that - back in the WM days we crammed in all sorts of silly flashcard functionality without thinking it through properly and the usability of the system has suffered as a result.

And frankly, if an option like this is buried in a settings screen somewhere and it's not immediately obvious what it does or how to use it, too few people will actually use it for it to be worth implementing; we'll have almost as many people enabling it by accident (or without realizing what it does) and getting ticked off that they're not learning anything (because they're actually not studying enough cards) as we will genuinely using it.

So pretty much all of our current efforts on the flashcard selection algorithm front are being made with an eye towards simplicity and ease-of-use; having too many flashcards due to study is certainly a problem, but we want to solve it in a way that will benefit every user of our flashcard system rather than just a few.

 

[HW60]

And frankly, if an option like this is buried in a settings screen somewhere and it's not immediately obvious what it does or how to use it, too few people will actually use it for it to be worth implementing

I would agree if that option would be an option, but chosing the number of cards to be reviewed is very simple to understand (much more simple than SRS itself) and should not be treated as an option, but as important input like the category to be reviewed.

So pretty much all of our current efforts on the flashcard selection algorithm front are being made with an eye towards simplicity and ease-of-use; having too many flashcards due to study is certainly a problem, but we want to solve it in a way that will benefit every user of our flashcard system rather than just a few.

I fully agree with simplicity and ease-of-use, and I have no problems with Pleco's flashcard system under "normal" conditions, but could you just give a slight hint as to the plans how to improve the system when too many cards arise from a pause like holidays/sickness or else? What is the intended solution to recover from such a situation? After 2 or 3 weeks without reviewing the required number of cards is beyond anything I would like to catch up. Therefore I cannot imagine a solution without reducing the number of cards to review. The most important cards to review in this situation are the cards with a low score, as Gato already pointed out. Therefore he proposed to use the score filter, starting with the lowest possible score, and going up when there is time to review. But increasing the upper limit of the score filter is certainly only a method for advanced users.

I use Gato's system since about 10 days now and feel very comfortable with it: I concentrate on reviewing the cards I do not know, climb up with the upper limit of the score filter, and the higher the upper limit gets the less important it becomes how many days the flashcard is beyond the due date of SRS. Instead of increasing the score filter selecting the number of cards would really increase simplicity and ease-of-use even for those who do not understand what a score or a score filter is.

 

[mikelove]

I would agree if that option would be an option, but chosing the number of cards to be reviewed is very simple to understand (much more simple than SRS itself) and should not be treated as an option, but as important input like the category to be reviewed.

The problem is that then people would misuse it - we could have easily added a "number of cards" option for spaced repetition a long time ago, but we didn't because SRS simply doesn't work if you keep such an option enabled all the time without doing anything else to manage the number of cards due for review.

I fully agree with simplicity and ease-of-use, and I have no problems with Pleco's flashcard system under "normal" conditions, but could you just give a slight hint as to the plans how to improve the system when too many cards arise from a pause like holidays/sickness or else? What is the intended solution to recover from such a situation? After 2 or 3 weeks without reviewing the required number of cards is beyond anything I would like to catch up. Therefore I cannot imagine a solution without reducing the number of cards to review. The most important cards to review in this situation are the cards with a low score, as Gato already pointed out. Therefore he proposed to use the score filter, starting with the lowest possible score, and going up when there is time to review. But increasing the upper limit of the score filter is certainly only a method for advanced users.

Well you don't actually need to use the score filter to get that effect - you can create a profile with the card selection system "fixed," sorting by score in descending order, and limit the number of cards in that. You'll know what the score range is like if you configure the score to show at the top of the screen.

But as I think I've already said a few times, the basic idea of what we're working on is to remove cards from active study if you get too far behind, so that you'll still be able to review the remaining cards often enough to remember them. We would not do this immediately after you'd been away for a few weeks - in fact in your scenario you'd get something pretty close to what you're talking about, a whole bunch of cards prioritized by score - but for somebody who simply turns this option on and leaves it that way, as a lot of people would, the only way we can offer any kind of meaningful card selection is to to cull their list of cards to study when they have too many behind schedule.

 

[HW60]

The problem is that then people would misuse it - we could have easily added a "number of cards" option for spaced repetition a long time ago, but we didn't because SRS simply doesn't work if you keep such an option enabled all the time without doing anything else to manage the number of cards due for review.

My proposal was:
- use SRS
- let the user choose the number of cards for the next review
- let Pleco manage the upper limit of the score filter
Then even with a moderate rate of correct answers of about 80% or more the upper limit increases with every review. The user does not even have to know about the way Pleco temporarily limits the number of cards. You can roughly calculate how long it will take that the upper limit of the score filter has reached the top score of the profile and thereby the "normal" SRS.

Well you don't actually need to use the score filter to get that effect - you can create a profile with the card selection system "fixed," sorting by score in descending order, and limit the number of cards in that. You'll know what the score range is like if you configure the score to show at the top of the screen.

I think that the card selection system "fixed" - that is exactly Gato's system - is less effective than using SRS for card selection, because with "fixed" you always repeat all cards within the limit range, while with SRS you only review the cards that are due (within the limit range), and therefore with SRS the speed with which the upper limit increases is greater.

But as I think I've already said a few times, the basic idea of what we're working on is to remove cards from active study if you get too far behind, so that you'll still be able to review the remaining cards often enough to remember them.

That sounds somewhat definite - how do you want to select the cards you want to remove? Is it necessary to remove cards only because you are behind? I think this looks like 2 different problems: The problem you describe is the user who tries to learn too many cards, though he is able to do his daily reviews. The problem I try to describe is being behind because you cannot do your daily reviews for a day or two or a week or two.

We would not do this immediately after you'd been away for a few weeks - in fact in your scenario you'd get something pretty close to what you're talking about, a whole bunch of cards prioritized by score - but for somebody who simply turns this option on and leaves it that way, as a lot of people would, the only way we can offer any kind of meaningful card selection is to to cull their list of cards to study when they have too many behind schedule.

I have no idea about your proposal to those people who are "away for a few weeks". Caused by holidays, the due cards in my flashcards are by far too many to catch up by "just reviewing". So unfortunately for those people who are at their limit with the daily amount of cards the case of being behind is the normal case! 1 or 2 days will do!

 

[mikelove]

The user does not even have to know about the way Pleco temporarily limits the number of cards. You can roughly calculate how long it will take that the upper limit of the score filter has reached the top score of the profile and thereby the "normal" SRS.

No, because people will enable that option and never turn it off, thus never catching up and eventually reviewing cards farther and farther behind schedule.

That sounds somewhat definite - how do you want to select the cards you want to remove? Is it necessary to remove cards only because you are behind? I think this looks like 2 different problems: The problem you describe is the user who tries to learn too many cards, though he is able to do his daily reviews. The problem I try to describe is being behind because you cannot do your daily reviews for a day or two or a week or two.

It's not that definite, just some scribblings in a notebook at this point. But you asked for a hint as to how we planned to improve the system and that's the direction we're looking at the moment.

I have no idea about your proposal to those people who are "away for a few weeks". Caused by holidays, the due cards in my flashcards are by far too many to catch up by "just reviewing". So unfortunately for those people who are at their limit with the daily amount of cards the case of being behind is the normal case! 1 or 2 days will do!

My tentative thinking is that we'll present people with an alert when they first run a flashcard test after several days away to let them decide how to handle the problem - remove cards from study, distribute extra cards over the next few days, etc.

 

[mandarin_student]

I'm an intermediate user who loves Pleco and I have been following this discussion very closely. I suspect my concerns are shared by many Pleco users: I'm not a tech expert, I do not enjoy spending lots of time reading forums, and I do not like having to tweak settings more than once (if at all). I simply want to learn Chinese efficiently and get on with other things in my life (for which I need to read, write, and speak Chinese). My concerns are the following:

1. I want the process of flashcard testing to be simplified, at least in the main menu. It is simply too time consuming to configure settings at the moment. A new selection of preset settings for a range of users and needs would be ideal.
2. I want to stay true to SRS principles. If not, as Mike suggests, I risk not learning characters properly -- which is the very reason I started using Pleco flashcards in the first place.
3. Like HW60 and many other users I know, I desperately need some way to "recover" from a period of time away from Pleco. From time to time, I will be unable to use Pleco for several weeks at a time. The backlog of SRS flashcards is impossible to overcome in large decks; and it makes no sense to split cards into separate smaller decks. So some way to limit the number of cards for the first few days after a long time away is crucial -- especially if Pleco is going to be something I use throughout the course of my life.
4. Following from the above point: I don't really care how cards are reduced, provided that there is some SRS or "weighted" logic to the reduction. Ideally, I would be able to pick a manageable number of cards (say 300) to be my upper limit for daily SRS revision until I have revised all overdue cards. I understand this poses problems as Mike explains. But something approximating this would work best for users like me who just want to get through what they missed over a few days/week, and then get back on track. I do not want to "remove cards from study" if this means they do not return! (I can already manually eliminate cards from further testing if and when they are truly learned -- a great function).
5. To respond to Mike's concern, I would not leave this "card reduction" function on forever! I can see how some users might forget or get lazy, preferring the reduced or capped card deck all the time. But serious users would not do this.
6. I would HATE to see the weighted repetition setting disappear. This is VERY useful. For example, when preparing for a test/speech/meeting etc., I always select a deck of relevant cards (such as chapter vocab or themes vocab like "economics") and use the weighted repetition review function to text myself on the cards I most often forget. This is a BRILLIANT way to brush up one's shirt term memory for a particular task. I'd really like to see this functionality preserved through some test setting or filer, and I think other users would find this useful for the same reasons.

I'd be very grateful if you could address these point in the forum and in the update (when is it due?). To reiterate, I am by not as tech savvy user who has time to understand all the details of flashcard settings -- time is precious and I want to spend it revising characters, not fine-tuning settings. I suspect this is why, Mike, you want to simplify the whole process, which I wholeheartedly endorse.

I just want SRS to be manageable for those occasions when life gets in the way!

 

[mikelove]

Addressing this point by point:


1-2) Yes, this is pretty much along the lines of how we're planning to improve flashcard configuration.

3-4) You might not want to remove them from study, but people who aren't putting a lot of thought into their flashcard configuration will be better off if we do that than if we add a manual option that still throws too many cards at them, or that can be configured in such a way that they'll never actually catch up. So we need to support some sort of removal behavior as a default, then provide an option for users to avoid having cards removed if they know what they're doing - otherwise we haven't really made things usefully simpler.

5) Enough users would do it that we need to leave it off by default, and default to a behavior through which you'll actually keep learning cards.

6) Some sort of "N highest score" or "N lowest score" card filter is something HW60 mentioned before and seems like it might be a good option here as well - "weighted" is simply too random to be genuinely useful.

To be frank, your post is a little bit self-contradictory in that you're simultaneously saying that you want everything simpler but also asking for more options / advanced user features; it's not really possible to satisfy your points 1/5/6 simultaneously. This is true for a lot of our users, however, and since the latter option would cause us to spiral off into ever-greater realms of Confusing Submenus we're pretty much stuck pursuing the former, even if it means upsetting a few people by taking things away.

So the basic path forward on flashcards - and this will probably mostly come in the next update after the current dictionary-and-reader-and-general-UI-focused Big Update - is to:

a) Remove rarely-used or difficult-to-continue-supporting options;
b) Streamline the configuration UI, both by taking away and more intelligently burying options;
c) Redesign the SRS system to better handle tricky cases like long delays between reviews automatically; and
d) Add new features (of course).

 

[gato]

Anki 2 appears to have added a "per day" review limit, which defaults to 100, to handle the situation of backlogs.

http://ankisrs.net/docs/changes.html#perday
Decks now have a limit on daily reviews as well, which defaults to 100. When the limit is reached, Anki will not present further reviews from that deck, even if there are cards waiting. When the congratulations page is reached, a notice will be displayed, advising the user that for optimum memory, they should consider increasing the daily limit. This change is intended to reduce the sense of being overwhelmed that people feel after returning to Anki from a break. Large due counts can be off-putting, and while it’s not optimal for a user to leave due cards waiting, if they give up in frustration that’s an even worse outcome.

http://ankisrs.net/docs/manual.html#falling-behind
Falling Behind

If you fall behind in your reviews, Anki will prioritize cards that have been waiting the longest. It does this by taking the 200 cards that have been waiting the longest, and showing them to you in a random order up until your daily review limit. This ordering ensures that no cards will be left waiting indefinitely, but it means that if you introduce new cards, their reviews won’t appear until you’ve gotten through your backlog.

If you wish to change the order of the overdue reviews, you can do so by creating a filtered deck, as described in a later section of the manual.

When you answer cards that have been waiting for a while, Anki factors in that delay when determining the next time a card should be shown. Please see the frequently asked questions section at the bottom of the manual for more information.

 

[mikelove]

Interesting that they have one, but that's the sort of approach I'm actually trying to avoid - the goal here is to be easier to use than Anki. If our fix for this problem likewise involves a warning message then we haven't really done our job.

There's a fundamental difference between Pleco and Anki in that the defining feature of our flashcard module is that it's integrated with our dictionary, not that it has a highly-customizable SRS system. Hence, to maximize its usefulness we want to have a flashcard module that can appeal to anybody who's interested in our dictionary, not just those who are both interested in our dictionary and in highly-customizable SRS.

So it should be possible for somebody to create some flashcards, tell our system how many cards they feel like reviewing right now, and have that review happen in an intelligent / optimized way without having to think about any of the other settings. That doesn't mean that our app can't still be highly customizable, but it does mean that whatever solutions we come up with for problems like falling behind on reviews have to be designed in such a way that people can potentially use them without having to think about them.

 

[gato]

Another reason I pointed out the change in Anki 2 is to show how commmon this backlog problem is to SRS -- for the traditional SRS algorithm assume that one study every days but in reality most people study spordically (if not all the time, then from time to time), and the flashcard system should allow for that. It's better to study sporadically than to totally give up and not study at all because the software is only usable by the most disciplined students.

Would you be williing to discuss the idea forward a little more in depth?

the basic idea of what we're working on is to remove cards from active study if you get too far behind, so that you'll still be able to review the remaining cards often enough to remember them. We would not do this immediately after you'd been away for a few weeks - in fact in your scenario you'd get something pretty close to what you're talking about, a whole bunch of cards prioritized by score - but for somebody who simply turns this option on and leaves it that way, as a lot of people would, the only way we can offer any kind of meaningful card selection is to to cull their list of cards to study when they have too many behind schedule.

You mentioned that the current idea to deal with backlogs is to (1) remove cards from the study list, (2) but not to remove them immediately.

This seems potentially very complicated and confusing to the user (i.e. non-transparent) depending on what you mean by "not immediately". Would you have a formula to determine how gradually to remove the cards?

Wouldn't it be easier to simply allow the user to set an upper limit to number of cards to study per day, so that if there is a backlog, the number the cards studied would be capped?

 

[mikelove]

To be honest I haven't worked it into a form that I can describe much more specifically yet - 95% of my design brain at the moment is tied up with the forthcoming not-particularly-flashcard-related-aside-from-OS-7-UI-and-hopefully-server-sync-and-some-modest-refinements-to-settings big iOS update.

But I don't necessarily see transparency as a requirement as long as we offer people the option to turn any not-particularly-transparent feature off. Removing cards is what we do for someone who doesn't want to think about this problem and just wants the system to keep showing them cards, and for that group I don't really see an alternative - you only want to study 100 flashcards a day, you've got a large backlog of cards that you're not going to get through at that rate, we can either put up an annoying message telling you to study more cards or we can simply quietly remove some cards from study and let you continue on your merry way.

So it seems to me like we could do that by default, but then offer people who want to personally manage their flashcard pool the option to turn it off and manage them in whatever way they like.

As far as what we can do for that group, to be honest the thing I'm getting stuck on is this insistence on setting a limit; the SRS system already prioritizes the most overdue cards when you start a new review, so if you just manually stop after 200 cards you'll have the exact same behavior that you would have had if we stopped automatically. So is it really so necessary that we add an that automatic stopping option? An option that's going to do more harm than good for anybody who doesn't know what they're doing with SRS? At least until we've got this new super-easy system working (and have buried most of the more advanced options off in a sub-subscreen with a warning the first time you try to use them) it seems like for the vast majority of users it'll only make things worse.

 

[natecfields]

Hi Mike,
I previously posted a question on another post about exporting my pleco flashcards from Pleco to Anki. However I noticed that Anki is super complex with only four videos and a very very long handbook to read so I know what to do. Before I knew what I was doing I was looking up code and they said it shouldn't be too complicated which only made me more discouraged because I still couldn't understand. Anyway, I thought maybe I could just see if I am maximizing the study features I have with pleco before I move on to add another system. I have alot of paper flashcards I use for grammar apart from the vocab flashcards I review for Pleco. The benefit I found to making cards on Anki is I thought I could help structure and quiz myself on certain patterns and the guys at Hacking Chinese really spoke highly of Anki, that is from posts I saw they posted from 2012. I really use Pleco daily, just like the function of making some of my own flashcards and also reviewing so many of the cards I have stashed in decks I don't really think to have reviewed. Do you have any preferred settings for reviewing flashcards in your Pleco decks that you feel maximize your study? thanks!

I am going to be needing to add several new vocabulary terms related to special education/physcial therapy and would love to have a synced way of storing and studying new words. Thanks!

Nathan

 

[mikelove]

At the moment I'd say the defaults for "Spaced Repetition" are pretty much my ideal.

But honestly one of the biggest problems we're hoping to address this year is that people get the sense the settings *aren't* optimal - some of this is because our algorithm needs to do a better job of adjusting to people automatically (you shouldn't have to think about optimal settings, we should figure them out for you), but we think also that we need to do a better job of paring down our options; actually, one option we've been considering is eliminating most of them altogether but instead giving users the option to code their own custom interval calculation algorithm in JavaScript if they don't like ours, so we'd replace a couple of dozen options with just two or three of them plus a "use custom javascript" box.

 

[gato]

actually, one option we've been considering is eliminating most of them altogether but instead giving users the option to code their own custom interval calculation algorithm in JavaScript if they don't like ours, so we'd replace a couple of dozen options with just two or three of them plus a "use custom javascript" box.

Isn't that going from one extreme to the other? It would be make changes in phases, even if minimalism is eventually where you would like to end up. Perhaps also focus more on the other often requested changes in the first phase, like adding support for sound/recording, improving support for sentences, making it easier to edit cards, etc, simplifying options in general (not just for the card selection options).

 

[mikelove]

It's more that I'd rather not go to the effort of supporting lots of old, little-used options in a new system. Having too many options tends to make people assume that whatever options they start off with are less-than-optimal - it puts a burden onto the user to configure everything correctly that we ought to be taking on ourselves. And I'm not currently convinced that burying a lot of options in an 'advanced' screen will adequately address that.

Certainly working on other things too, but since algorithm/database stuff are at the lowest level we're addressing those first - more logical to build later stuff on that than to build it on top of old code that we're about to replace.