Cantonese spellcheck with Metaphones

672 views

Published on

full study on http://www.corgitoergosum.net/studies/

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
672
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cantonese spellcheck with Metaphones

  1. 1. CANTONESE SPELL CHECKING WITH METAPHONES / full study on http://www.corgitoergosum.net/studies/Tuesday, 14 June 2011
  2. 2. a study to see if this sort of spell suggestion is possible: query: query: Do you mean“ ”? query:Tuesday, 14 June 2011
  3. 3. Unlike more formal pingyin methods, the “HK Govt Cantonese pingyin”, commonly used in street names and identification papers in HK, is based on English pronunciation...Tuesday, 14 June 2011
  4. 4. In other words, two Cantonese words should have cantonese pingyins that sound similar in English “fung hei yue”Tuesday, 14 June 2011
  5. 5. query: “ ”Tuesday, 14 June 2011
  6. 6. query: “ ” look up pronunciation table pronunciation: “luk ding gei”Tuesday, 14 June 2011
  7. 7. query: “ ” look up pronunciation table pronunciation: “luk ding gei” search phrase dictionary for similar sounding words suggestion: Do you mean“ ”?Tuesday, 14 June 2011
  8. 8. pronunciation table pingyin char ban ban ban ban ban banTuesday, 14 June 2011
  9. 9. using metaphone algorithm to compute “sound hashes” (chan yik suen) XNYKXN XNYKXN XNYKSN XNLKSN XNYXN XNSNKTuesday, 14 June 2011
  10. 10. finding similar sounding words: the levenshtein distance algorithm distance from XNYKXN XNYJNK 3 XNYKSN 1 XNLKSN 2 XNY_XN 1 XN_SNK 4 XNYTKWN 2 XMYTSNK 4 YTYMSYPFT 8Tuesday, 14 June 2011
  11. 11. seems like one is the safest from this limited test set distance from XNYKXN XNYJNK 3 XNYKSN 1 XNLKSN 2 XNY_XN 1 XN_SNK 4 XNYTKWN 2 XMYTSNK 4 YTYMSYPFT 8Tuesday, 14 June 2011
  12. 12. some test results query suggestion no matches no matchesTuesday, 14 June 2011
  13. 13. some test results query suggestion no matchesTuesday, 14 June 2011
  14. 14. some test results query suggestionTuesday, 14 June 2011
  15. 15. limitations no tonal info HK Govt pingyin is very loose doesn’t consider similar shaped characters, like: , , , , ......Tuesday, 14 June 2011
  16. 16. improvement areas speed only looks at of first character now english is ignored use libyell dict both for segmentation and common words (n- grams) checking simplified chinese rewriteTuesday, 14 June 2011
  17. 17. feedback?Tuesday, 14 June 2011

×