How the machine understands Korean 기계와 대화를 하려면 어떻게 해야 할까요? 우리는 그 동안 기계가 이해할 수 있는 프로그래밍 언어를 만들어서, 그 언어를 통해 소통해 왔습니다. 하지만 2010년 들어서며 급물살을 탄 AI 연구는 이러한 소통의 영역까지 침투하여, 기계가 인간의 언어를 이해하고, 소통할 수 있는 단계로 다가서고자 노력하고 있습니다. 그 근간에는 선형대수학의 여러 이론들이 사용되고 있는데요, 특히 인간의 언어를 기호화하고 이를 벡터공간에 투영하는 방법들이 핵심으로 여겨지고 있습니다. 이러한 방법을 임베딩(embedding)이라 지칭하고, 단어부터 문장, 문서에 이르기까지 인간의 언어를 다양한 형태로 벡터화하고, 이를 이용해 언어의 의미 유사성, 관계 유사성 등을 벡터 공간에서 벡터 연산을 통해 내재적인 의미를 도출합니다. 이번 세미나에서는 벡터공간모델(Vector Space Model, VSM)의 전통적인 방법(TF-IDF, SVD 등)부터 신경망 방법(word2vec, sent2vec 등)에 이르는 다양한 언어 모델링들을 살펴보고, 이를 한국어에 적용했을 때 기계가 어떻게 의미를 이해하는 것으로 해석할 수 있는지 다양한 관점에서 실험을 통해 살펴보도록 하겠습니다.

2. 2

3. a mechanically, electrically, or electronically operated device
for performing a task
https://www.merriam-webster.com/dictionary/machine3

4. a process by which information is exchanged
between individuals through a common system
of symbols, signs, or behavior
https://www.merriam-webster.com/dictionary/communication4

5. 5 https://www.youtube.com/watch?v=stM8dgcY1CA

6. 6
NOT GATE
AND GATE
OR GATE

7. 7 https://www.electronics-tutorials.ws/boolean/bool_8.html

8. 8 https://kldp.org/node/110850
A = [0, 0, 0, 1, 1, 0, 0, 1]
B = [0, 1, 1, 1, 0, 0, 0, 0]
S = [1, 0, 0, 0, 1, 0, 0, 1]

9. 9

10. 10 https://theasciicode.com.ar/

11. 11
>>> ord("a")
97
>>> ord("A")
65
>>> ord(" ")
54620
>>> ord(" ")
44397
>>> ord(' ')
50612
>>> hex(ord(' '))
'0xac00'
https://unicode.org/charts/PDF/UAC00.pdf

12. 12 https://www.youtube.com/watch?v=P5KS7F4Javk
Encoding

13. 13
Decoding

14. 14

15. 15
Encoding
Decoding

16. 16
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xac150xc5440xc9c0'

17. 17
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xac150xc5440xc9c0'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xcc3e0xc5440xc918'

18. 18
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xac150xc5440xc9c0'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xcc3e0xc5440xc918'

19. 19
Task
Output
19

20. 20

21. 21 https://ithub.korean.go.kr

22. 22

23. 23

24. 24

25. 25
0xcf54 0xc5b4 0xb2f7 ...
0xcf54 0xc5b4
0xb2f7
0xc5d0 0xc11c

26. 26
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xac150xc5440xc9c0'

27. 27

36. 36

37. 37

38. 38
J(A, B) =
|A B|
|A [ B|
=
|A B|
|A| + |B| |A B|<latexit sha1_base64="+8ZkZSL4zCU4I4Ifhr2eq4IJGew=">AAADHHicjVHLSsNAFD3G97vq0s1gESpqSVTQjaB1I64UrBVaKcl0qsE0CclEkKa/4p+4cyduFdduFPQfvDNG8EHRCUnOnHvOmbkzTui5sTTNxx6jt69/YHBoeGR0bHxiMjc1fRQHScRFmQdeEB07diw81xdl6UpPHIeRsFuOJyrO+Y6qVy5EFLuBfygvQ3HSsk99t+lyWxJVz1XYXmF7qbTANlm7nW6zGrdDVko7tYBcH0SiiU43RcoWWVpK2TL7UuzUc3mzaOrBfgMrA3lkYz/IPaCGBgJwJGhBwIck7MFGTE8VFkyExJ2gTVxEyNV1gQ5GyJuQSpDCJvacvqc0q2asT3OVGWs3p1U8eiNyMsyTJyBdRFitxnQ90cmK7Zbd1plqb5f0d7KsFrESZ8T+5ftU/tenepFoYkP34FJPoWZUdzxLSfSpqJ2zL11JSgiJU7hB9Ygw187Pc2baE+ve1dnauv6slYpVc55pE7yoXdIFWz+v8zc4Wilaq8WVg7X8Vim76iHMYg4Fus91bGEX+yhT9jWe8Io348q4MW6Nuw+p0ZN5ZvBtGPfvKkqvhQ==</latexit>
https://en.wikipedia.org/wiki/Jaccard_index

39. 39

40. 40

41. 41 Christopher, D. M., Prabhakar, R., & Hinrich, S. (2008). Introduction to information retrieval. An Introduction To Information Retrieval, 151(177), 5.

42. 42 Christopher, D. M., Prabhakar, R., & Hinrich, S. (2008). Introduction to information retrieval. An Introduction To Information Retrieval, 151(177), 5.

43. 43 Christopher, D. M., Prabhakar, R., & Hinrich, S. (2008). Introduction to information retrieval. An Introduction To Information Retrieval, 151(177), 5.
• Each term !" generates a row vector ($"%, $"', ⋯ , $"))
referred to as a term vector and each document +, generates a
column vector
+, =
$%,
⋮
$/,

44. 44 Christopher, D. M., Prabhakar, R., & Hinrich, S. (2008). Introduction to information retrieval. An Introduction To Information Retrieval, 151(177), 5.
A =
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
1 1 0 0
1 0 1 0
1 1 1 0
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
0 0 0 1
0 0 0 1
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
<latexit sha1_base64="CnY+57CJKvSKGuwemxFFRmUiI9c=">AAADjnicjVHJTsMwEJ0QlhK2AEcuFhWIU0kKAi4VBS4cQaIUiZQqcU2xmk2Og6iqfiBfgPgD+AvGJmVH4MjJ85v3JjOeIA15Jh3n0Rgzxycmp0rT1szs3PyCvbh0niW5oKxBkzARF4GfsZDHrCG5DNlFKpgfBSFrBr0jFW/eMpHxJD6T/ZS1Ir8b82tOfYlU274/IDXiBazL40Ea+VLwu6HlknWitqO25xFrdHA/Eu5Xwhmp/iJc8j+L8xPxzlgeiztvVbftslNx9CLfgVuAMhTrJLEfwIMOJEAhhwgYxCARh+BDhs8luOBAilwLBsgJRFzHGQzBQm+OKoYKH9kevrt4uizYGM8qZ6bdFP8S4hboJLCGngR1ArH6G9HxXGdW7G+5Bzqnqq2P36DIFSEr4QbZv3wj5X99qhcJ17Cne+DYU6oZ1R0tsuT6VlTl5ENXEjOkyCncwbhATLVzdM9EezLdu7pbX8eftFKx6kwLbQ7PqkocsPt1nN/BebXiblWqp9vl+mEx6hKswCps4Dx3oQ7HcAINoMam0TCujLZpmztmzdx/lY4ZhWcZPi3z+AWY8LZs</latexit>

45. 45 https://nlp.stanford.edu/IR-book/essir2011/pdf/vspace.pdf

46. 46
cos(dj, q) =
dj · q
kdjk kqk<latexit sha1_base64="WEJvkGAkLSXB3SDtYKm070hlEcc=">AAADRXicjVFNS+QwGH5bvz/WHZX14iU4CArL0HEFvQiyXvbogqOClSHNpDNx2qaTpoLUwv67/QmLf0D04G3Z6/omVt11EE1p++b5St4kSCORac+7ctyR0bHxicmp6ZnZD3Mfa/MLh5nMFeMtJiOpjgOa8UgkvKWFjvhxqjiNg4gfBf09wx+dc5UJmRzoi5SfxrSbiFAwqhFq1374MdU9FZOCyawka512cVZ+HqzvFH6oKCOF5YOQFJYpic86UpMndFCSsvAjHmr/ckiqRLdn8Bf04Jkqy3at7jU8O8hw0ayKOlRjX9Z+gQ8dkMAghxg4JKCxjoBChs8JNMGDFLFTKBBTWAnLcyhhGr05qjgqKKJ9/HZxdlKhCc5NZmbdDFeJ8FXoJLCKHok6hbVZjVg+t8kGfS27sJlmbxf4D6qsGFENPUTf8j0q3+szvWgIYdv2ILCn1CKmO1al5PZUzM7JP11pTEgRM3UHeYU1s87HcybWk9nezdlSy99YpUHNnFXaHG7NLvGCmy+vc7g43Gg0vzQ2vm/Wd79WVz0Jy7ACa3ifW7AL32AfWph97cw6n5wl96d75/52/zxIXafyLMJ/w/17DwePwrE=</latexit>
=
PN
i=1 ai,jai,q
qPN
i=1 a2
i,j
qPN
i=1 a2
i,q<latexit sha1_base64="gEOiazfmNk7ySnU2NhATa9UfnvM=">AAADN3icjVHNTtwwGPw2FNhSoAsce4m6qsQBrZKlElyQEFx6QiB1AYmFlWO8YMjfOg4SivJgPAntqTcER16g6tgYqS3ix1Hi8XwzE392lMey0EHws+GNvRufmGy+n/owPTP7sTU3v1tkpeKix7M4U/sRK0QsU9HTUsdiP1eCJVEs9qLzTVPfuxCqkFn6XV/m4jBhJ6kcSs40qEFLrVX9oWLcr/pFmfiDSq6F9VG1VTPApbOHaVTXFeojpZ+THVXdGqIXNCOnqetBqx10Ajv8pyB0oE1ubGeta+rTMWXEqaSEBKWkgWNiVOA5oJACysEdUgVOAUlbF1TTFLwlVAIKBvYc3xOsDhybYm0yC+vm+EuMV8Hp0xd4MugUsPmbb+ulTTbsc9mVzTR7u8QcuawErKZTsK/5HpVv9ZleNA1p1fYg0VNuGdMddymlPRWzc/+vrjQScnAGH6OugLl1Pp6zbz2F7d2cLbP1W6s0rFlzpy3pzuwSFxz+f51PwW63Ey53ujtf2+sb7qqb9Ik+0yLuc4XW6RttUw/ZP+h3Y7LR9K68X96Nd/sg9RrOs0D/DO/+D/sovjI=</latexit>

47. 47
A =
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
1 1 0 0
1 0 1 0
1 1 1 0
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
0 0 0 1
0 0 0 1
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
<latexit sha1_base64="CnY+57CJKvSKGuwemxFFRmUiI9c=">AAADjnicjVHJTsMwEJ0QlhK2AEcuFhWIU0kKAi4VBS4cQaIUiZQqcU2xmk2Og6iqfiBfgPgD+AvGJmVH4MjJ85v3JjOeIA15Jh3n0Rgzxycmp0rT1szs3PyCvbh0niW5oKxBkzARF4GfsZDHrCG5DNlFKpgfBSFrBr0jFW/eMpHxJD6T/ZS1Ir8b82tOfYlU274/IDXiBazL40Ea+VLwu6HlknWitqO25xFrdHA/Eu5Xwhmp/iJc8j+L8xPxzlgeiztvVbftslNx9CLfgVuAMhTrJLEfwIMOJEAhhwgYxCARh+BDhs8luOBAilwLBsgJRFzHGQzBQm+OKoYKH9kevrt4uizYGM8qZ6bdFP8S4hboJLCGngR1ArH6G9HxXGdW7G+5Bzqnqq2P36DIFSEr4QbZv3wj5X99qhcJ17Cne+DYU6oZ1R0tsuT6VlTl5ENXEjOkyCncwbhATLVzdM9EezLdu7pbX8eftFKx6kwLbQ7PqkocsPt1nN/BebXiblWqp9vl+mEx6hKswCps4Dx3oQ7HcAINoMam0TCujLZpmztmzdx/lY4ZhWcZPi3z+AWY8LZs</latexit>
cos(d1, d2) =
2
2.83 ⇥ 1.41
= 0.5
<latexit sha1_base64="Mz3fqZdI6Gmx11iq+hTEdN8ueuA=">AAAC8nicjVHLShxBFD12XmoetrrMpsgQmEDSdLcG3QiimywVHEeYGYbumppJYb+oqhZkmK9w5y5kmx9wqx8h/oH5i9yqtJBkCEk13X3q3HtO1b03rTKpTRjeLXiPHj95+mxxafn5i5evVvzVtWNd1oqLDi+zUp2kiRaZLETHSJOJk0qJJE8z0U1P9228eyaUlmVxZM4rMciTSSHHkieGqKH/gZe6PRpG79loGL9jO6w/VgmfxrNpHGxv9I3MhWZRsBnNdsLg49BvhUHoFpsHUQNaaNZB6d+ijxFKcNTIIVDAEM6QQNPTQ4QQFXEDTIlThKSLC8ywTNqasgRlJMSe0ndCu17DFrS3ntqpOZ2S0atIyfCWNCXlKcL2NObitXO27N+8p87T3u2c/mnjlRNr8JnYf+keMv9XZ2sxGGPb1SCppsoxtjreuNSuK/bm7JeqDDlUxFk8orgizJ3yoc/MabSr3fY2cfF7l2lZu+dNbo3v9pY04OjPcc6D4ziINoL4cLO1u9eMehGv8QZtmucWdvEJB+iQ9wWucI0bz3iX3hfv689Ub6HRrOO35X37AVvnnlE=</latexit>
[[1.0 , 0.5 , 0.5 , 0.67 ],
[0.5 , 1.0 , 0.5 , 0.0 ],
[0.5 , 0.5 , 1.0 , 0.0 ],
[0.67 , 0.0 , 0.0 , 1.0 ]]

48. 48
d1 d2 d3
w1 1 0 0
w2 0 1 0
w3 1 1 1
w4 1 1 0
w5 0 0 1
-0.27 0.21 0.70 -0.53 0.30
-0.27 0.21 -0.70 -0.53 0.30
-0.71 -0.33 0 -0.10 -0.60
-0.55 0.43 0 0.64 0.29
-0.15 -0.77 0 0.10 0.60
2.35 0 0
0 1.19 0
0 0 1.00
0 0 0
0 0 0
-0.65 0.26 0.70
-0.65 0.26 -0.70
-0.36 -0.92 0
=

49. 49

50. 50
A0
=
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
0.95 0.54 0.54 0.04
0.95 0.54 0.54 0.04
1.23 0.8 0.8 0.18
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.26 0.22 0.22 0.8
0.26 0.22 0.22 0.8
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
<latexit sha1_base64="7xtN193IeVqY4dyiGQFiKzwdqM0=">AAADwXictVHLTuswEJ0Qnr08CizZRFQ8NkROaKEskHhsWIJ0C0gUocQ1xWpechwEqtizhZ9D/AH8AwvGrovgogo215Enx2fmjGc8YRbxXBLybA3ZwyOjY+MTpT+TU9Mz5dm54zwtBGUNmkapOA2DnEU8YQ3JZcROM8GCOIzYSdjZV/6TayZyniZ/5W3GzuOgnfBLTgOJ1EX5bXfF2XaaIWvzpJvFgRT85q5E3K3aMnFr1b4h1WZzAOu5/jriut5rxPXqvVBFko2e8VxS+6+sv6Gu9n1jsZRBNEtaH31elCuYRi/nO/AMqIBZh2n5CZrQghQoFBADgwQk4ggCyPE7Aw8IZMidQxc5gYhrP4M7KKG2wCiGEQGyHbRtPJ0ZNsGzyplrNcVbItwClQ4soSbFOIFY3eZof6EzK3ZQ7q7OqWq7xX9ocsXISrhC9iddP/K3OtWLhEuo6x449pRpRnVHTZZCv4qq3PnUlcQMGXIKt9AvEFOt7L+zozW57l29baD9LzpSsepMTWwBr6pKHLD37zi/g2Pf9dZd/6ha2dkzox6HBViEVZznJuzAARxCA6gVWvfWg/Vo79vczmzRCx2yjGYeviy7+w7I98cy</latexit>
[[ 1. , 0.67 , 0.67 , 0.71],
[ 0.67, 1. , 1. , -0.05],
[ 0.67, 1. , 1. , -0.05],
[ 0.71, -0.05, -0.05, 1. ]]

51. 51
A0
=
0
B
B
B
B
B
B
B
B
B
B
B
B
B
B
@
0.95 0.54 0.54 0.04
0.95 0.54 0.54 0.04
1.23 0.8 0.8 0.18
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.93 0.06 0.06 1.05
0.26 0.22 0.22 0.8
0.26 0.22 0.22 0.8
1
C
C
C
C
C
C
C
C
C
C
C
C
C
C
A
<latexit sha1_base64="7xtN193IeVqY4dyiGQFiKzwdqM0=">AAADwXictVHLTuswEJ0Qnr08CizZRFQ8NkROaKEskHhsWIJ0C0gUocQ1xWpechwEqtizhZ9D/AH8AwvGrovgogo215Enx2fmjGc8YRbxXBLybA3ZwyOjY+MTpT+TU9Mz5dm54zwtBGUNmkapOA2DnEU8YQ3JZcROM8GCOIzYSdjZV/6TayZyniZ/5W3GzuOgnfBLTgOJ1EX5bXfF2XaaIWvzpJvFgRT85q5E3K3aMnFr1b4h1WZzAOu5/jriut5rxPXqvVBFko2e8VxS+6+sv6Gu9n1jsZRBNEtaH31elCuYRi/nO/AMqIBZh2n5CZrQghQoFBADgwQk4ggCyPE7Aw8IZMidQxc5gYhrP4M7KKG2wCiGEQGyHbRtPJ0ZNsGzyplrNcVbItwClQ4soSbFOIFY3eZof6EzK3ZQ7q7OqWq7xX9ocsXISrhC9iddP/K3OtWLhEuo6x449pRpRnVHTZZCv4qq3PnUlcQMGXIKt9AvEFOt7L+zozW57l29baD9LzpSsepMTWwBr6pKHLD37zi/g2Pf9dZd/6ha2dkzox6HBViEVZznJuzAARxCA6gVWvfWg/Vo79vczmzRCx2yjGYeviy7+w7I98cy</latexit>
[[ 1. , 0.67 , 0.67 , 0.71],
[ 0.67, 1. , 1. , -0.05],
[ 0.67, 1. , 1. , -0.05],
[ 0.71, -0.05, -0.05, 1. ]]

52. 52 https://serimag.com/en/nlp-machines-managed-to-understand-us/

53. 53 http://www.joanechilds.com/services/nlp-hypnotherapist/

54. 54

55. 55

56. 56
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xac150xc5440xc9c0'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xace00xc5910xc774'
>>> hex(ord(" "))+hex(ord(" "))+hex(ord(" "))
'0xcc3e0xc5440xc918'

57. 57 https://en.wikipedia.org/wiki/John_Rupert_Firth

58. 58
(맥도날드가, 햄버거는)
(맥도날드가, 맛있다.)
(맛있다., 맥도날드가)
(맛있다., 감자튀김도)
(감자튀김도, 맛있다.)
(감자튀김도, 맛있었는데..)
(맘스터치도, 햄버거는)
(맘스터치도, 맛있다.)
(맛있다., 맘스터치도)
(맛있다., 패티가)
Source Text
Red : Target keyword, Blue : Context Keyword
Training Set

59. 59
(맥도날드가, 햄버거는)
(맥도날드가, 맛있다.)
Input, Output

60. 60

61. 61 https://ronxin.github.io/wevi/

62. 62

63. 63

64. 64

65. 65

66. 66

67. 67

68. 68

69. 69

70. 70

71. 71

72. 72

73. 73
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , |
, | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | , | ,
| , | , | , | , | , | , | , | , | , | , | , |

74. 74

75. 75

76. 76

77. 77

78. 78

79. 79

80. https://projector.tensorflow.org/80

81. 81

82. 82

83. 83

84. 84

85. 85 Neural Networks for NLP, Tomas Mikolov

86. 86 Neural Networks for NLP, Tomas Mikolov

87. 87 Neural Networks for NLP, Tomas Mikolov

88. 88
[(0.8520864248275757, ' '),
(0.761702299118042, ' '),
(0.7503935694694519, ' '),
(0.7462027072906494, ' '),
(0.7302743792533875, ' '),
(0.7298693656921387, ' '),
(0.7157598733901978, ' '),
(0.7140597105026245, ' '),
(0.7094160318374634, ' '),
(0.6926915049552917, ' ')]

89. 89
[(0.8487251996994019, ' '),
(0.8287239074707031, ' '),
(0.8190956115722656, ' '),
(0.8059816956520081, ' '),
(0.8007813692092896, ' '),
(0.7956226468086243, ' '),
(0.7848511934280396, ' '),
(0.7843033671379089, ' '),
(0.7841789722442627, ' '),
(0.7816827297210693, ' ')]

90. 90
[(0.8593053221702576, ' ')
(0.8095390796661377, ' ')
(0.7830708026885986, ' ')
(0.759726881980896, ' ')
(0.7565611004829407, ' ')
(0.750198245048523, ' ')
(0.7494476437568665, ' ')
(0.7444630861282349, ' ')
(0.7321089506149292, ' ')
(0.730089545249939, ' ')]

91. 91
[(' ', 0.6118873953819275),
(' ', 0.6057026386260986),
(' ', 0.6024502515792847),
(' ', 0.6006665229797363),
(' ', 0.5892309546470642),
(' ', 0.5832505822181702),
(' ', 0.57846599817276),
(' ', 0.5780129432678223),
(' ', 0.5749800205230713),
(' ', 0.5698598623275757)]

92. 92
[(' ', 0.718355119228363),
(' ', 0.7033782005310059),
(' ', 0.6210535764694214),
(' ', 0.618556022644043),
(' ', 0.6083796620368958),
(' ', 0.6076724529266357),
(' ', 0.5991458892822266),
(' ', 0.5892307758331299),
(' ', 0.5869563817977905),
(' ', 0.5819442272186279)]

93. 93
[(' ', 0.7236467599868774),
(' ', 0.7141597270965576),
(' ', 0.7086147665977478),
(' ', 0.6981553435325623),
(' ', 0.6899087429046631),
(' ', 0.6880921125411987),
(' ', 0.6837730407714844),
(' ', 0.6807584166526794),
(' ', 0.6780474185943604),
(' ', 0.6770625114440918)]

94. 94
[(' ', 0.6854178309440613),
(' ', 0.6564003229141235),
(' ', 0.6439071297645569),
(' ', 0.6154448986053467),
(' ', 0.6112699508666992),
(' ', 0.6107276082038879),
(' ', 0.608704149723053),
(' ', 0.6080746650695801),
(' ', 0.6069726347923279),
(' ', 0.6008787155151367)]

95. 95
[(' ', 0.7925821542739868),
(' ', 0.777511477470398),
(' ', 0.7687333822250366),
(' ', 0.768500804901123),
(' ', 0.7665073871612549),
(' ', 0.763087809085846),
(' ', 0.7591485381126404),
(' ', 0.7579624056816101),
(' ', 0.7577899098396301),
(' ', 0.7568272352218628)]

98. 98

99. 99

100. 100

101. 101 http://docs.likejazz.com/sent2vec/

102. 102

103. 103 https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77
https://miro.medium.com/max/928/1*kvcBEC6in6UYS4J3Im311w.gif

104. http://docs.likejazz.com/bert/104

105. 105 http://docs.likejazz.com/bert/

106. 106 https://www.upwork.com/hiring/for-clients/artificial-intelligence-and-natural-language-processing-in-big-data/

107. 107 https://www.youtube.com/watch?v=xAFrKKApHTY

108. 108 Nickel, Maximillian, and Douwe Kiela. "Poincaré embeddings for learning hierarchical representations." Advances in neural information processing systems. 2017.

109. 109