Cantonese spellcheck with Metaphones

CANTONESE SPELL CHECKING WITH METAPHONES / full study on http://www.corgitoergosum.net/studies/Tuesday, 14 June 2011

a study to see if this sort of spell suggestion is possible: query: query: Do you mean“ ”? query:Tuesday, 14 June 2011

Unlike more formal pingyin methods, the “HK Govt Cantonese pingyin”, commonly used in street names and identiﬁcation papers in HK, is based on English pronunciation...Tuesday, 14 June 2011

In other words, two Cantonese words should have cantonese pingyins that sound similar in English “fung hei yue”Tuesday, 14 June 2011

query: “ ”Tuesday, 14 June 2011

query: “ ” look up pronunciation table pronunciation: “luk ding gei”Tuesday, 14 June 2011

query: “ ” look up pronunciation table pronunciation: “luk ding gei” search phrase dictionary for similar sounding words suggestion: Do you mean“ ”?Tuesday, 14 June 2011

pronunciation table pingyin char ban ban ban ban ban banTuesday, 14 June 2011

using metaphone algorithm to compute “sound hashes” (chan yik suen) XNYKXN XNYKXN XNYKSN XNLKSN XNYXN XNSNKTuesday, 14 June 2011

ﬁnding similar sounding words: the levenshtein distance algorithm distance from XNYKXN XNYJNK 3 XNYKSN 1 XNLKSN 2 XNY_XN 1 XN_SNK 4 XNYTKWN 2 XMYTSNK 4 YTYMSYPFT 8Tuesday, 14 June 2011

seems like one is the safest from this limited test set distance from XNYKXN XNYJNK 3 XNYKSN 1 XNLKSN 2 XNY_XN 1 XN_SNK 4 XNYTKWN 2 XMYTSNK 4 YTYMSYPFT 8Tuesday, 14 June 2011

some test results query suggestion no matches no matchesTuesday, 14 June 2011

some test results query suggestion no matchesTuesday, 14 June 2011

some test results query suggestionTuesday, 14 June 2011

limitations no tonal info HK Govt pingyin is very loose doesn’t consider similar shaped characters, like: , , , , ......Tuesday, 14 June 2011

improvement areas speed only looks at of ﬁrst character now english is ignored use libyell dict both for segmentation and common words (n- grams) checking simpliﬁed chinese rewriteTuesday, 14 June 2011

feedback?Tuesday, 14 June 2011

Cantonese spellcheck with Metaphones

by kenshin03 on Jun 14, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Cantonese spellcheck with Metaphones — Presentation Transcript