Rethinking User Dictionaries

[mikelove]

So as part of the 3.2 database updates we're thinking about making a big and potentially controversial change: basically, we would replace the current user dictionary system with two new things:

• Dictionary integration of custom flashcards - you could now search for custom flashcards just as for any other dictionary entries. Specific flashcard categories (along with 'all flashcards') would be add-able to custom dictionary groups if you wanted to narrow down your search to only some of them.
• Non-editable user-created dictionary databases; you could dump a bunch of flashcards to one of these and could probably also create it directly from a text file. But it would be a one-time thing, fixed / immutable once you'd created it (and gaining a lot of performance optimizations through that).

Couple of clear benefits to this:

• Much easier sync, both on our end (only have to sync your flashcard database) and on yours (no more wondering whether a particular user dictionary has synced / fiddling around with how to sync it / etc).
• No more juggling dictionary entries versus flashcards - anything that you're editing / working on is in your flashcard database.
• Easier access to your custom flashcards (you no longer have this weird database of useful information you've collected that's walled off from the rest of the app).
• Vastly easier development for us since we're no longer dealing with two separate user-editable database formats.

But we also imagine it'll disrupt some people's workflows pretty severely, hence this post - is there anything about this plan that's likely to conflict with how you use Pleco?

Migration-wise, we would most likely turn all of your existing user dictionaries into static dictionaries but then give you an easy way to dump all of the entries from any user dictionary into a new flashcard category.

 

[Shun]

I think it would be a great simplification, I can't see any negative side-effects. Those people who use user dictionaries are slightly more advanced users anyway, so they should be able to adapt to the new system pretty quickly.

I would also welcome dictionary-based flashcards' categories to show up in a search, perhaps using a tap-hold bubble option. Would that also be easier to accomplish using the new design? (perhaps in v3.3?)

 

[Abun]

for me personally that sounds less convenient than the current system. I use user dicts either (1) to digitalize shorter glossaries of words (in this case I add a whole new dictionary) or (2) to add words which I couldn't find in the professional dicts. This mainly includes slang expressions, expressions from local languages which swept into Putonghua (the Taiwanese kiū for example, which describes a certain kind of 口感), rarer variants of Chengyu and so on (in this case I use one seperate user dict to which I add all of these entries). However in most cases I do not intend to add them to my list of flashcards to learn. (1) would unnecessarily flood my flashcards with hundreds of words, many of which I already know. (2) is often not added in order to really learn them actively but just for the sake of completeness. Especially with Chengyu variants I usually learn the most common variant and just want to save the rarer one in case I ever stumble across it and don't recognize it. For example, if I know 能言善道 I don't feel like I should learn to actively use 能言惯道 as well (also because this creates confusion as to which one I'm asked for when I tell pleco to ask me for headwords).

Plus, I'm not quite sure about the implemention in the search process. I like all of my entries available in one search, so I would regard it as a bit of a nuisance to first search the (professional) dicts and then run a flashcard search as well, just in case the word is in one of the custom dicts gone flashcard. This would especially be true because I use the HanDeDict quite a lot and since that is technically a user dict, it would get converted as well

 

[Shun]

Hi Abun,

You can put these unimportant phrases, like, for example, 能言惯道, in the new kind of user-created, non-editable dictionary database. So there wouldn't be any confusion, because the category you're learning only includes the chengyu 能言善道. If you put all words/phrases not intended for learning in one flashcard category and create sub-categories within it, they also won't get in the way.

The implementation of Search will be quite simple, a single search for both dictionaries and custom flashcards. You can freely pick which flashcards are searched by adding categories to Dictionary Groups. HanDeDict has always behaved like a normal dictionary, because you cannot change any of its entries from within Pleco.

 

[Abun]

Hello Shun,

as far as I understood the plan, the dictionary database would be created on implemention of the new system. After this point the dictionary would not be editable anymore (like the normal dictionaries). It would therefore work for the entries I will have created up to that point, but not for new things I might want to add at a later point. Creating more different categories of flashcards should work, but just for me personally that sounds like more work than I have with new entries now.

The problem might be the way in which I use custom dictionaries and flashcards. I don't upload my custom dictionaries for other people to download because most of the entries are not free of 3rd party rights. So I'm guessing there shouldn't be much to sync there. For the flashcards I don't bother with categories, I simply add any word I need to uncategorized and learn that. This way I have no way of predicting any card, which I find helps me internalize them better. This way of course, my flashcards are just my personal vocabulary list which includes all kinds of (previously) unknown words that I happen to come across without any kind of order. Therefore I never uploaded them, either. Consequently, simplifications in the syncing process for me personally seem to have little to no benefit, but I probably would feel the drawbacks, i.e. the (arguably not all that big) increase in work needed to manage custom entries and flashcards.

Does that mean that HanDeDict (and while we're at it MoE as well) is treated in just the same way as normal dictionaries, or as a locked custom one? If there is a difference at all of course.

 

[Shun]

Hi Abun,

yes, you could put the digitized glossaries in a non-editable user dictionary.

If you wish to learn flashcards without any way of predicting a card based on the category it's from, you can create a test with all categories except the category of cards you do not wish to learn. If I were you, I would just create a single flashcard category and put all Uncategorized cards in that category, as well as a category and sub-categories with cards you wish to use as a changeable user dictionary.

Sync can also be useful for backup or for sharing of flashcards among different devices.

MoE is a real user dictionary, because it was downloadable from the forum and had to be installed as a user dictionary.

 

[alex_hk90]

So as part of the 3.2 database updates we're thinking about making a big and potentially controversial change: basically, we would replace the current user dictionary system with two new things:

• Dictionary integration of custom flashcards - you could now search for custom flashcards just as for any other dictionary entries. Specific flashcard categories (along with 'all flashcards') would be add-able to custom dictionary groups if you wanted to narrow down your search to only some of them.
• Non-editable user-created dictionary databases; you could dump a bunch of flashcards to one of these and could probably also create it directly from a text file. But it would be a one-time thing, fixed / immutable once you'd created it (and gaining a lot of performance optimizations through that).

How much of a performance gain would the latter be over the current user dictionary system and then 'locking' it? Once I locked the database, I don't recall having any performance issues with any of the user dictionaries I have installed, even when I was using my old HTC Desire.

Not being able to edit user dictionaries once imported feels like a step backwards to me, but to be fair I very rarely have edited my user dictionaries (maybe only a couple of times when I noticed an error) so it wouldn't be too much of a hassle to edit the source and re-import the whole thing for these rare occasions.

 

[santiago]

personally I would be very happy with the change. being able to look at the flashcards as if they were a dictionary is something that I always thought was missing in pleco. And frankly speaking, finding an option "missing" in pleco's flashcard system is a remarkable achivement haha!!!!
cheers, amazing product!!.

 

[HW60]

I have only custom cards and create a new user dictionary made of all flashcards about every 2 weeks. About half of my flashcards are japanese. There are several problems with Japanese flashcards and more problems with japanese user dictionary entries, e.g. search for Kana headwords or Kana pronunciation. I would be happy if some of These problems would be solved, and very unhappy if some of the very good existing solutions would not work after the db change.

 

[bglasow]

I think it would be a good simplification, a great integration of the flashcards into the core product, and would reduce a lot of the confusion between the two competing approaches that currently exist.

 

[HW60]

It might be a good idea to solve the duplicate cards problem in flashcards with 3.2: cards with different values in the headwords (or no traditional headword) and different pinyin should be treated as different cards and not as duplicates (to be modified by settings).

 

[Bendy-Ren]

I think it sounds great.

Would that mean any time you edit a flashcard that was previously just a dictionary entry—say, rewording the definition—that the new version would show up in a dictionary search alongside the default entries?

 

[gabor]

Creating user dictionaries was the #1 reason I bought Pleco and an iPad 2 years ago. I have created over 12 000 entries altogether and the only reason I stopped is because the way it worked was still not good enough for my needs. Instead I started coding it myself, just the user dictionary part. Other than that I really love Pleco, I use the text reader most often, now that I stopped adding entries to my user dictionaries. So, while I am curious how the update is going to turn out I dont have much hope that I can stop programming my own solution

 

[mikelove]

Thought I'd let this sit for a bit before replying since I didn't want to seem to be cutting off discussion, but since it's been about a month now:

@Shun - actually that's pretty much the idea, custom entries in flashcards would be no different from user dictionary entries search-wise now. (at least optionally - could still hide them if you wanted to)

@Abun - I think we'd be importing your user dictionary entries to flashcards, actually; you could essentially use flashcard categories in the way that you would currently use separate user dictionary databases, and then dump a particular category / set of categories to a more permanent database for separated (and perhaps slightly faster) search and sharing.

@alex_hk90 - there would be some performance increase, but not much. We might potentially even still allow editing in these standalone dictionaries, at least of definitions if not headwords - the change would more impact the various places in Pleco's UI where you might create a custom dictionary definition now (reader selections, search results, edit buttons at the top of dictionary entries, etc), those would all be geared towards this single flashcard database.

@HW60 - understood; I think in your case this might actually even end up working better, since your custom flashcards would now show up as custom dictionary entries without your having to keep exporting / re-generating your user dictionary. Regarding duplicate management, the tentative plan is to make that customizable - we have a lot of people with traditional-only or simplified-only custom cards who would like the system to accommodate them better too.

@Bendy-Ren - yes, exactly that.

@gabor - could you be any more specific about where our system is lacking?

 

[HW60]

That sounds like a real improvement, merging the 2 databases for flashcards and user dictionary.

I do not know if it is a matter of database, but there is a very old idea of creating character flashcards automatically for every flashcard word, e.g. if I enter a flashcard 学生, Pleco should create 2 flashcards 学 and 生 (if they do not exist until then), or Pleco should at least check which characters are not yet created by the user if an automatic creation should be too difficult ...

 

[mikelove]

That's been sitting in our to-do list forever, for the sake of the stroke order feature if nothing else - still pending, just have to get around to doing it.

 

[Taichi]

I'm not sure if this is appropriate place to ask, but I often happened to want to highlight part of dictionary. As currently theres no way to do that, I copy-paste that part to create user dictionary entry (or flashcard). Is there any plan to support this kind of usecase?

 

[mikelove]

We're working on a 'notes' feature, in fact the Edit Entry button that shows up at the top of the screen if you've created a user dictionary is an early version of that (since it lets you easily add a user definition that will then come up on subsequent searches for the same word). But nothing specifically for highlighting - what are you trying to achieve by highlighting parts of entries? Are you isolating a particular aspect of a word that you want to study, or ask questions about, or make a note to yourself about, or something else?

 

[Taichi]

There are two cases I want to highlight parts of entries.
- isolating particular meaning of word. For example, I know what 就 means, but recently found it has special meaning (GH 9. 菜肴､果品等搭配着主食或酒吃). If it were a paper dictionary, I would highlight that part.
- highlight an particular example sentence. I want to highlight useful sentences or sentences I'm not comfortable with.

Implementing highlights might be tricky as dictionaries can be updated.
Maybe you can add something like "append to note" button when user selected the text.

 

[ACardiganAndAFrown]

I think for user dictionaries more flexible pinyin 'systems' need to be installed.

like @Abun above adding in Taiwanese entries & @gabor above saying it doesn't meet his needs - I posted this back in 2011 and I don't think much has changed since then: http://www.plecoforums.com/threads/user-dictionary.2926/

I even wrote an email to Mike a while back saying:
"...I tried using it with Pleco's user dictionary but ran into many problems along the way (un-standardized pinyin being the biggest, which affected searching and displaying of pronunciations, etc) so in the end I decided to write my own app..." (similarly to @gabor)

@Abun's kiū wouldn't work either..

CC-CEDICT also has problems with this:

the above entry is completely 'unsearchable' through pinyin search...

If user dictionaries are going to be useful I think this is a big thing to consider.