Tracking Flash Card Status

[paulwilt]

Hi,

First of all let me say I am a dedicated Pleco user and a strong advocate (my lastest verbiage is "...it goes without saying that you want this dictionary; every student I have talked to, whether geek or technophobe, calls it the most important learning tool they have...")

What I am wondering is whether Pleco would consider adding some functionality to the flash cards. Specifically what I am looking for is an additional way to manage the words; in my desktop software (an Access application based on the CEDICT dictionary) I call it a "Task Basket."

The idea behind a Task Basket is that we can't neatly divide the characters we are studying into "Know/Don't Know" or "Got Right/Got Wrong." The fact is, some characters I know what they mean, but I forgot how they're pronounced (Hanzi > English = Pass, but Hanzi > Pinyin = Fail). Some words I know how to say, but I don't know the character (English > Pinyin = Pass, English > Hanzi = Fail).

With Chinese, there are three main elements of information: Characters; Pinyin/Pronunciation; "meaning" (for me, English)...let's call them "Cues," since they are the candidates for presentation on a flash card.

Pleco's flash card functionality, like many flash card programs, allows me to control which Cues I want to see and which will be hidden (we can call the hidden ones "Answers"). I can change which elements are Cues and which ones are Answers each time I start a flash card session.

What the program does not (yet?) allow me to do is:
1. Define multiple different Cue/Answer pairings ("Tasks")
2. Put Tasks in order; that is, consistently promote words from one Task to the next
3. Group words together based on their membership in a Task (which represents how well the have been learned, and is almost certainly different from their membership in a Lesson, since some members of the Lesson are learned immediately, while others stick in one's craw for weeks).

This is the functionality I first implemented in my vocab management program. It allows you to define a Task, which includes a set of Cues and corresponding Answers...for example:
Task 0: Show Character and Pinyin, Expect Meaning
Task 1: Show Character, Expect Pinyin and Meaning
Task 2: Show Meaning, Expect Pinyin
Task 3: Show Pinyin, Expect Character
Task 4: Show Meaning, Expect Character

...as you can see, this set of Task definitions takes the learner from character recognition (an easy task) to character production (a much more difficult task), with a short stop at oral production (the Meaning > Pinyin step). I have found this to be a very natural progression and would recommend it to new learners (though I think it should be implemented in a way that each user can define and change).

Since the Tasks are ordered from easy to hard, it makes sense to promote vocabulary items from Task 0 to 1 to 2 to 3...and so on. For this reason I also call the Tasks "Bins," as at any given time each Task may "contain" a number of "cards." Note that the vocab items in any given Bin (that is, at any given stage in the learning process) may have come from different Lessons. This reflects the simple fact that we learn some items in each Lesson faster than others.

Although this seems like a very fundamental change (indeed, for the learner it may amount to a fundamentally different approach to flash-card learning), from a database perspective it does not involve that much data. Each item which has been added to a Vocabulary List would have a number representing its current Bin (Task); and each Task would have to be defined as including one or more Cues (which are already implemented, since the program already allows the user to choose which Cues to show for each flash card session).

Note that the Task Basket implementation saves you the trouble of tracking which items you "Got Right" and "Got Wrong." "Got Right/Wrong" doesn't map onto any real learning stage anyway, since whether you get an item right or wrong depends on what cues are available, which you can change with each flash card session. If you get it right, promote to next Bin. If you got it wrong, keep it in its current Bin. If you quiz yourself on each Bin each day, your words will slowly (some more slowly than others!!) march toward mastery.

Finally, this single change enables another very powerful and interesting capability. No matter how many Bins a user defines (I used to use ten but have cut back to six), the top bin represents words the user considers "learned." Counting the items in this bin, then, represents the user's active vocabulary! (if the Bins are defined and ordered carefully, one of the lower Bins should represent the user's passive vocabulary). This is a great motivational tool and it makes a cool graph.

I hope that you will consider this idea, which has done wonders for my character-learning and I am sure could help many other learners. How many times have you tested yourself on a Lesson, wasting 50% of your time slogging through words you knew you would get right, all to test yourself on the 50% you hadn't yet mastered? But you couldn't delete those items from the Lesson...if only you could get them out of the way by promoting them to another stage of learning, which would be represented by a different challenge/response drill (a different set of cues, i.e. a different "Bin").

If Task Bins is a lot to implement in your flash card utility (which admittedly is a sideshow to your indispensable dictionary), I hope you will consider importing from .txt files rather than .pdb for non-Palm-literate developers like me! I could export my current "Bins" to .txt in no time flat.

Thanks for a great program, thanks in advance for considering this idea, and thanks to all for laboring through a long long post!!

PHW

PS Any time Pleco People are in Hai Dian the pizza is on me. PHW

 

[mikelove]

Thank you so much for that very useful and well-thought-out suggestion; we've gotten a few other requests for a sort-of "ranking" system for flashcard entries, but this adds some interesting new dimensions to it.

What I'm thinking at this point is that the ideal system would treat these as sort of an ordered list of categories - there'd be a list somewhere where you could add, edit, delete, and re-order these "Tasks", and for each Task you could customize the dictionary fields displayed, the ones requested, how many correct answers were required to advance to the next task (and whether or not they have to be all in a row), how many incorrect answers would fall back to the previous task (or optionally all the way back to the first one), and maybe a few other things too. You could also view a list of all the flashcards in *any* word list that were grouped under this "Task" and all of the flashcards in a particular word list that were grouped under it. We would also include a set of presets, so that if all you wanted to do was have entries come up less often the more times you recognized it correctly - adding a little customizability to your Task-based system allows us to accommodate almost any conceivable mechanism for tracking flashcard performance.

And .txt file imports are almost a definite yes, very easy to do and something a *lot* of people would like (and we need to implement a large chunk of that functionality anyway just to convert old flashcard data files to the new format)

Thanks again for that great suggestion, and if you (or anyone else) have/has any more thoughts on this please do let me know.

Michael Love
Pleco Software

 

[paulwilt]

Structure of a Task/Bin/Rank

Hello,

Thanks for the response, I thought I would sketch out some of the fields I found useful when implementing Tasks.

Your description is right on, we want to create a way of organizing the vocabulary that is independent of its source (word list/lesson). This second way of organizing the data is based on how well the learner is doing at mastering it. Level, Bin, Task, Rank, whatever.

Some of the things I have found useful in tracking:

- TaskIndex: puts the task in order relative to any others that might be defined. I start at 0, like the other reader, because when we add things to the list we don't know them at all. The average user might be scared bu a zero-based array though. In any case, words progress from low-numbered Tasks to higher ones.

- Cues: This, of course, is what makes the Task work, this is a list of which cues are visible.

- Answer: I implemented this as a single answer (not the inverse of the set of Cues), but I like your idea of requiring multiple answers (whichever elements are not Cues!).

On the desktop, I implemented the flash card always using the same layout, just hiding any controls that were not Cues in the current Task (Bin). This made the design super convenient; everything is in the same place for every Task (it's just that some of it is not visible). When the user claims to have answered the question, you just unveil the hidden controls. But you might not have the screen real estate to do it this way on the Palm, haven't thought about that.

- QuestionType: I do multiple-choice questions on the desktop, but what I call "Offline" questions (where the user has to answer a dialog asking whether the answer was right or wrong) would probably be fine for a portable Palm app (again, screen real estate).

- TimeLimit: sometimes it's more challenging and appropriate if the Cues disappear after a second or two (user-defined property of the Task)

- PassToTaskIndex and FailToTaskIndex: although in general I pass to the next bin and failed words stay in their current bin, sometimes it's convenient to say "Pass bin 0 to 6" or "Fail Bin 6 to 0". For instance, when you inherit someone else's lesson (there will be a lot of this going on after you implement importing/exporting/beaming lessons!), you might want to zoom through it clearing it of things you know that you know.

The other ideas -- only pass after n correct responses, only pass n times in m hours, are related to the log, which I put in another post which appears to have vaporized. That's sad because it had my ideas about "Problem Children" and their definitions.

Anyway, if you are interested you might want to look at the ontology I set up I the Access database itself, you can download it and some more arguments about how to set up flash card systems at
http://www.paulwilt.com/Chinese/Tools/F ... g/main.asp
http://www.paulwilt.com/Chinese/Techniq ... acters.htm

Finally, I agree that you should definitely have an easy-to-follow default setup that installs automatically. Too many bins will just freak out new users. You do want to progress from passive to active, so you could have default definitions like:
0: Show Character (Expect pinyin and English)
1: Show Pinyin (Expect English)
2: Show English (Expect Pinyin and Character)

I think that using this system for just a few days, everyone would get the hang of the idea and start thinking about splitting some of the tasks. If you don't split them, whichever answer you find harder is likely to hold you back from getting a right answer. And everybody likes getting things right.

A side benefit to having more tasks is that I would rather see a word once a day for five days than five times in one day. More tasks help ensure that the word hangs out in your system for a little longer.

Hope this finds you well and hope I recover that post about logging and Problem Children!
PHW

 

[mikelove]

Thanks for following up on this. As there have not been very many computer-based Chinese flashcard systems released to date it's extremely useful to hear about how an real, working, developed and tested system other than ours works.

I don't think we'd bother assigning indexes to tasks at all (well we would, but not in a user-perceivable way) - the convention on Palm OS is to simply have a reorderable list. It's a lot more intuitive and more in line with what people are expecting. And you're correct that we don't really have the screen space on Palm to simply hide and show controls on the layout (well we do now, but we won't with all of these extra fields and other geegaws we're adding) - however we're already going to have to design a mechanism for handling this in the main dictionary view (so people can show and hide fields in that) and it'll be trivial to apply it to flashcards. Offline questions seem to work well enough for us for now (or at the very least nobody's complained about them), so we'll probably hold off on doing multiple-choice until we get a lot of requests for it.

I'm sorry that that post disappeared - did the confirmation page come up saying that it had been submitted? Logging a lot of data can get problematic on a slow, low-memory PDA, but at a bare minimum we'll certainly be keeping track of right/wrong answers for each word, and it probably makes sense to put some sort of a timestamp in there as well.

Actually I don't even think we would necessarily need to impose 'tasks' at all in some default setups - some people prefer to switch back and forth between tasks for all their cards at once, e.g. someone about to take a pinyin review test. All they want is for the flashcard system to keep track of how they're doing and show words they get correct less often, and of course we want to accommodate them too. (we are after all trying to sell something here) But from a "learning methodology" standpoint I certainly agree with you that many if not most people would be a lot better off splitting up their memorization into specific tasks; as an oft-frustrated Chinese learner myself I certainly would have benefitted

Anyway, I've downloaded your databases and will be taking a good look at them later on, and I hope you won't mind if I contact you with a few follow-up questions at some point.

Thanks again,

Michael Love
Pleco Software

 

[paulwilt]

Problem Children

Thanks again,

Well I would hardly call FlashBang "tested" but maybe you can get some ideas from it. Hope so.

I never did get the Problem Children post back, after I hit Submit IE hung, and I hadn't saved the text anywhere else.

Essentially it's what you were talking about -- by adding a time stamp to the "right/wrong" log, you can enable a lot of very interesting and pedgogically effective queries to find and review items that aren't moving. Some interesting Problem Children definitions I use are:

Got wrong in last n hours
Got wrong n times in last m hours
Have not gotten right in n hours
Have looked up more than n times in last m hours (days)
Have looked up in more than one Lesson (added to more than one list)

With a log, the normal flash card rule "show me only items that I haven't gotten right yet during this quiz session" is just a specific case of "show me items I haven't gotten right since time t", a query which has a lot of uses besides just running through the stack once.

As I admitted in the Lost Post, keeping a log is obviously more difficult in the tiny Palm than in a relational database, and I honestly don't know anything at all about Palm development. That said, the most powerful log queries still really only need a few days worth of data, so you could purge it that frequently if it would help.

Take care and feel free to drop me a line if FlashBang generates any questions (or fails to behave!!)

PHW