Linguistic Regularities in Continuous Space Word Representations
WHY?
Vector space word representations capture syntactic and semantic regularities in language well.
Scalable Distributed DNN Training Using Commodity GPU Cloud Computing
WHY?
Synchronization is an important issue is distributed SGD. Too few synchoronization among nodes causes unstable training while too frequent synchoronization causes high communication cost.
Neural Word Embedding as Implicit Matrix Factorization
WHY?
Skip-Gram Negative Sampling(SGNS) showed amazing performance compared to traditional word embedding methods. However, it was not clear where SGNS converge to.
Dependency-Based Word Embeddings
WHY?
Traditional continuous word embeddings based on linear contexts. In other words, word embeddings considered only surrounding words as context.
Large Scale Distributed Deep Networks
WHY?
Models with huge number of parameters or huge amount of data do not fit in GPU memory of a machine.
Improving Distributional Similarity with Lessons Learned from Wrod Embeddings
WHY?
Word embedding using neural network(Skipgram) seems to outperform traditional count-based distributional model. However, this paper points out that current superiority of word2vec is not because of the algorithm itself, but because of system design choices and hyperparameter optimizations.
Hadamard Product for Low-rank Bilinear Pooling
in Studies on Deep Learning, Deep Learning
WHY?
Bilinear model can caputure rich relation of two vectors. However, the computational complexity of bilinear model is huge due to its high dimensionality. To make bilinear model more applicable, this paper suggests low-rank bilinear pooling using Hadamard product.
Stacked Attention Networks for Image Question Answering
in Studies on Deep Learning, Deep Learning
WHY?
Visual question answering task is answering natural language questions based on images. To solve questions that require multi-step reasoning, stacked attention networks(SANs) stacks several layers of attention on parts of images based on query.
