The document discusses the knowledge discovery process and provides insights into representations of data, including one-hot and distributed representations. It highlights various resources and guides for implementing neural network models and APIs, particularly for word2vec, LSTM, CNN, and RNN. Additional links are provided for further reading and tools related to the subject matter.

1. Summer 2017
Elvis Saravia
PhD, Information Systems and Applications
ellfae@gmail.com
Github username: omarsar
Questions: sli.do (#Z217)

2. 2

3. ●
●
●
●
●
●
● Knowledge Discovery (KDD) Process
3

4. 4

5. 5

6. ConceptNet
6

7. ●
●
●
7

8. Motel = [0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]
Hotel = [0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0]
●
●
One-hot representation
8

9. hotel = [0.728 0.234 -0.23 0.223]
Distributed representation (low-dimension vector)
9

10. 10
Paper source: https://arxiv.org/pdf/1301.3781.pdf

11. 11
Paper source: https://arxiv.org/pdf/1301.3781.pdf
Feedforward Neural Net Language Model (NNLM)
variables to optimize
denotes window range

12. 12

13. 13
P(the|over)
P(fox|over)
P(jumped|over)
P(the|over)
P(lazy|over)
P(dog|over)
P(VOUT
| VIN
)
How to define this prob. distribution?
Determines similarity in [-1,1]
Get a probability in [0,1] out of a similarity in [-1,1]

14. 14

15. 15
https://www.healthvault.com/en-us/health-bo
t/

16. 16

17. ● https://goo.gl/ppHX65
●
○ Gensim guide for word2vec: https://goo.gl/i2UrdH
● https://goo.gl/7b72S9
●
● https://goo.gl/uNJDrs
●
17

18. 18

19. 19

20. 20

21. 21

22. 22

23. 23

24. ● https://goo.gl/KYacjz
●
●
●
●
●
● https://goo.gl/JezgYg
●
24

25. a. Build API: (Flask/Django recommended)
b. Pretrained models: (Guide: https://goo.gl/5qt2Ki)
c. Visualization: d3js / plotly / tensorboard
a. LSTM - (Guide: http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
b. CNN - (Guide: https://goo.gl/PgLUs7)
c. RNN - (Guide: https://goo.gl/5L9kci
a. Starting point:https://rare-technologies.com/word2vec-tutorial#app
25