# Linguistic Regularities in Sparse and Explicit Word Representations

## WHY?

Vector offset method is used for word analogy task.

## WHAT?

The objective function of vector offset method can be interpreted as similarity in direction(PairDirection). Also, this objective can be re-interpreted as addition of 3 cosine similarity(3CosAdd). These two objectives show different performance. Since PairwiseDirection does not take into account the spatial distance between individual vectors, overall performance was worse than 3CosAdd.

argmax_b*(cos(b*, b - a + a*))\\
= argmax_b*(cos(b* - b, a* - a))\\
= argmax_b*(cos(b*, b) - cos(b*, a) + cos(b*, a*))

However, 3CosAdd objective is not good for balancing among different aspect of similarity. To deal with this issue, this paper suggest 3CosMul as an objective function.

= argmax_b*\frac{cos(b*, b)cos(b*, a*)}{cos(b*, a) + \epsilon})

## So?

Objective function with 3CosMul showed improved performance than 3CosAdd expecially with explicit representation.

## Critic

Large improvement in performance simply by replacing objective fuction.

Powered by aiden