## WHY?

Using external memory as modern computer enable neural net the use of extensible memory. This paper suggests Differentible Neural Computer(DNC) which is an advanced version of Neural Turing Machine.

## WHAT?

Reading and writing in DNC are implemented with differentiable attention mechanism.

The controller of DNC is an variant of LSTM architecture that takes an input vector($x_t$) and a set of read vectors($r_{t-1}^1,...,r_{t-1}^R$) as input(concatenated). Concatenated input and hidden vectors from both previous timestep($h_{t-1}^l$) and from previous layer($h_t^{l-1}$) are concatenated again to be used as input for LSTM to produce next hidden vector($h_t^l$). Hidden vectors from all layers at a timestep are concatenated to emit an output vector($\upsilon_t$) and an interface vector($\xi_t$). The output vector($y_t$) is the sum of $\upsilon_t$ and read vectors of the current timestep.

THe interface vectors are consists of many vectors that interacts with memory: R read keys($\mathbf{k}_t^{r,i}\in R^W$), read strengths($\beta_t^{r,i}$), write key($\mathbf{k}_t^w\in R^W$), write strength($\beta_t^w$), erase vector($\mathbf{e}_t\in R^W$), write vector($\mathbf{v}_t\in R^W$), R free gates($f_t^i$), the allocation gate($g_t^a$), the write gate($g_t^w$) and R read modes(\mathbf{\pi}_t^i).

Read vectors are computed with read weights on memory. Memory matrix are updated with write weights, write vector and erase vector.

Memory are addressed with content-based addressing and dynamic memory allocation. Contesnt-based addressing is basically the same as attention mechanism. Dynamic memory allocation is designed to clear memory as analogous to free list memory allocation scheme.