Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (8)
Similar a Statistical Semantic入門 ~分布仮説からword2vecまで~
Similar a Statistical Semantic入門 ~分布仮説からword2vecまで~ (16)
Más de Yuya Unno
Más de Yuya Unno (20)
Último
Último (20)
Statistical Semantic入門 ~分布仮説からword2vecまで~
- 1. 2014/02/06 PFI Statistical Semantic ~ word2vec Preferred Infrastructure (@unnonouno) ~
- 4. Semantics
- 9. Statistical Semantics Distributional Semantics ! ! !
- 10. [Evert10] NAACL2010 Stefan Evert Semantic Models Distributional
- 11. ??? [Evert10]
- 13. dog [Evert10]
- 14. (Distributional Hypothesis) The Distributional Hypothesis is that words that occur in the same contexts tend to have similar meanings (Harris, 1954). (ACL wiki ) ! !
- 15. (Statistical Semantics) Statistical Semantics is the study of "how the statistical patterns of human word usage can be used to figure out what people mean, at least to a level sufficient for information access” (ACL wiki ) ! !
- 17. [ 13] ! ! !
- 18. ! ! ! ! ! ! ! !
- 21. : Latent Semantic Indexing (LSI), Latent Semantic Analysis (LSA) [Deerwester+90] ! ! ! !
- 23. LSI ! ! ! ! SVD
- 28. ! ! !
- 29. Probabilistic Latent Semantic Indexing (PLSI) [Hofmann99] ! LSI ! ! ! ex: LSI
- 31. Latent Dirichlet Allocation (LDA) [Blei03] PLSI ! PLSI LDA !
- 32. LDA ! NLP ! ! 1
- 35. Non-negative Matrix Factorization (NMF) [Lee +99] ! SVD ! ! [Lee+99]
- 36. NMF = PLSI [Dinga+08] ! NMF PLSI ! NMF PLSI
- 37. Non-negative Tensor Factorization (NTF) [Cruys10] 3 ! ! 2 3
- 38. ! ! SVD
- 40. Neural Network Language Model (NNLM) [Bengio +03] ! ! N NN N-1
- 41. Recurrent Neural Network Language Model (RNNLM) [Mikolov+10] ! t-1 t ! NNLM N ! ! http://rnnlm.org
- 45. Skip-gram [Mikolov+13b] : w1, w2, …, wT ! wi c vw w 5
- 46. !
- 47. [Mikolov+13c] !
- 53. ! NN ! ! ! ! ! !
- 55. 1 ! ! ! ! ! [Bird+10] Steven Bird, Ewan Klein, Edward Loper, , , . . , 2010. [ +96] . . , 1996. [Evert10] Stefan Evert. Distributional Semantic Models. NAACL 2010 Tutorial. [ 13] . . , 2013. [Deerwester+90] Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, Richard Harshman. Indexing by Latent Semantic Analysis. JASIS, 1990.
- 56. 2 ! ! ! ! ! [Hofmann99] Thomas Hofmann. Probabilistic Latent Semantic Indexing. SIGIR, 1999. [Blei+03] David M. Blei, Andrew Y. Ng, Michael I. Jordan. Latent Dirichlet Allocation. JMLR, 2003. [Lee+99] Daniel D. Lee, H. Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. Nature, vol 401, 1999. [Ding+08] Chris Ding, Tao Li, Wei Peng. On the equivalence between Non-negative Matrix Factorization and Probabilistic Latent Semantic Indexing. Computational Statistics & Data Analysis, 52(8), 2008. [Cruys10] Tim Van de Cruys. A Non-negative Tensor Factorization Model for Selectional Preference Induction. Natural Language Engineering, 16(4), 2010.
- 57. 3 ! ! ! ! NN 1 [Bengio+03] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin. A Neural Probabilistic Language Model. JMLR, 2003. [Mikolov+10] Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan "Honza" Cernocky, Sanjeev Khudanpur. Recurrent neural network based language model. Interspeech, 2010. [Mikolov+13a] Tomas Mikolov, Wen-tau Yih, Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. HLT-NAACL, 2013. [Mikolov+13b] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. CoRR, 2013.
- 58. 4 ! ! ! NN 2 [Mikolov+13c] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, Jeffrey Dean. Distributed Representations of Words and Phrases and their Compositionality. NIPS, 2013. [Kim+13] Joo-Kyung Kim, Marie-Catherine de Marneffe. Deriving adjectival scales from continuous space word representations. EMNLP 2013. , [Mikolov+13d] Tomas Mikolov, Quoc V. Le, Ilya Sutskever. Exploiting Similarities among Languages for Machine Translation. CoRR, 2013.