WHY?

Vector offset method is used for word analogy task.

WHAT?

The objective function of vector offset method can be interpreted as similarity in direction(PairDirection). Also, this objective can be re-interpreted as addition of 3 cosine similarity(3CosAdd). These two objectives show different performance. Since PairwiseDirection does not take into account the spatial distance between individual vectors, overall performance was worse than 3CosAdd.

$$argmax_b*(cos(b*, b - a + a*))\\ = argmax_b*(cos(b* - b, a* - a))\\ = argmax_b*(cos(b*, b) - cos(b*, a) + cos(b*, a*))$$

However, 3CosAdd objective is not good for balancing among different aspect of similarity. To deal with this issue, this paper suggest 3CosMul as an objective function.

$$= argmax_b*\frac{cos(b*, b)cos(b*, a*)}{cos(b*, a) + \epsilon})$$

So?

Objective function with 3CosMul showed improved performance than 3CosAdd expecially with explicit representation.

Critic

Large improvement in performance simply by replacing objective fuction.

Levy, Omer, and Yoav Goldberg. “Linguistic regularities in sparse and explicit word representations.” Proceedings of the eighteenth conference on computational natural language learning. 2014.