Classical chinese poetry automation and chinese text analysis

1. Classical Chinese Poetry Automation
and
Chinese Text Analysis
Minghui Yu

2. Agenda
1. About me
2. Classical Chinese poetry automation
• Classical Chinese poetry features
• Challenges
• Progresses
3. Chinese text analysis
• Chinese vs English
• As a research tool

3. About me
•Minghui Yu
- Programmer Analyst, UBC IT (Mon-Fri, 9 AM- 5PM)
- Classical Chinese poetry enthusiast, amateur poet poem
writer (rest of the day)
- MBA (SFU,2016)

4. Childhood dream Today’s me

5. Poetry writing: Art or Science
v.s.

6. Classical Chinese Poetry Features
Conciseness
 Usually short
 i.e.: many are as short as only 20 Chinese characters
Forms(*)
 Rhyming
 Tone
 Vocabulary
*Regulated verse, since from early Tang Dynasty (618–907)

7. A poem example
(L) O L L O
红 豆 生 南 国
L L O O L
春 来 发 几 枝
(O) L L O O
愿 君 多 采 撷
O O O L L
此 物 最 相 思
Tone Pattern:
(L/O)-O-L-L-O
L-L-O-O-L
(L/O)-L-L-O-O
(L/O)-O-O-L-L
L: Level tone
O: Oblique tone
(L/O): Can be either
Rhyming pattern:
枝 Zhi 思 Si
相思 Miss You
Red beans in the South grow.
Spring comes, sprays aglow.
Pick more beans, may thou?
Of love they're the best show.

8. Challenges
• Syntax (easy)
 Tone (level or oblique tone)
 Rhyme (i.e.: Zhi, Si)
• Semantics (harder)
 Spring Autumn comes, sprays aglow?
• Pragmatics (hardest)
 If Autumn, then perhaps
chrysanthemum?
 Is chrysanthemum a good metaphor
for love?

9. Progress
Word Salad
- non-sense “poem”
- Syntax okay
Computer Aided; Template
based
- More like an advanced
proof reading/editing tool
Statistical Modeling(*)
- Case based reasoning
- Phrase-based
- Sentence Poem Block
Machine Learning(*):
- Recurrent neural networks
- Genetic algorithm
*Both use extensive statistical concepts and techniques; there is no
such a clear boundary.

10. Goal
Input Processing… Output
Topic:
Sense of urgency (忧患意识)
Keywords:
Great Wall (长城)
Dilapidated (破落)
Rhyme:
Level tone: Zhi (上平四支)
Tone:
(O)OLLO,
LLOOL.
(L)LLOO,
(O)OOLL .
Statistical Modeling
Machine Learning
四海承平久,
长城日渐危.
明朝胡犯我,
御虏又凭谁?
The country has been peaceful for
long,
The Great Wall is dilapidated and
broke down,
Shall enemies pour forces on,
Whom I can depend upon？
By the way, I wrote this poem, not computer 

11. Not easy
• 向晚黄鹂隔叶鸣
• 唤回残梦两三声
• 向晚: early evening/later afternoon
• 残梦: messy dreams (memory of dreams)
Messy dreams in the early evening/late afternoon?
Sounds like a semantic error.

12. But doable
白鹭窥鱼立
Egrets stood, peeping fishes
青山照水开
Water was still, reflecting mountains
夜来风不动
The wind went down by nightfall,
明月见楼台
as the moon came up by the tower.
Proceedings of the 2014 Conference on Empirical Methods in Natural Language
Processing (EMNLP), pages 670–680
Question:
An egret is any of several herons. Some
herons hunt in the night.
But do egrets hunt in the night?
If egrets do, this poem is good.
Otherwise, it has a semantic error too.

13. Chinese text analysis
Chinese VS English text analysis
Similarities
Differences

14. Source text: 我叫余明辉我老家是安徽省我住在Richmond市
Source text: My name is Yu Minghui My hometown is Anhui Province I live in City of Richmond

15. As a research tool
• Literature study
• Help researchers search certain topics more efficiently
• Compare literatures
• Journalism
• Business
• … …
Very similar to English text analysis as a research tool
There are more similarities than dissimilarities

16. Last but not least
Curiosity
Aspiration
Beauty
Application
Why Classical Chinese poetry
automation?
 The pursuit of beauty
 A journey of curiosity
 …
文章本天成
妙手偶得之 (陆游)
A great piece of work was created
by the Heaven,
A skilled person got it by chance
But
Can an algorithm got it by design?
Once again: poetry writing: Art or
Science?

17. Thank you!

18. Some resources
Open Source Natural Language Processing (NLP) framework:
• Stanford CoreNLP: https://stanfordnlp.github.io/CoreNLP/
Some Chinese NLP application demos:
• http://bosonnlp.com/
• https://cloud.baidu.com/product/nlp.html
• http://thulac.thunlp.org
• http://www.ltp-cloud.com/
• http://nlp.qq.com/