• Design of Digital Circuits

    Summary of Design of Digital Circuits course by Onur Mutlu in ETH Zurich. Thank you very much for opening up the great lectures and materials for self-learners like me. This course provided me invaluable insight to understand computers. Introduction and Basics Mysteries in Comp Arch Meltdown and Spectre RowHammer Introduction...


  • Distributed Prioritized Experience Replay

    WHY? Gorila framework separated several actors and learners with a centralized parameter server to parrallelize the learning process. This framework required one GPU per learner. WHAT? Ape-X architecture only consists of two parts: many actors and one learner. Given a model from model, many actors generate experience simultaneously. The learner...


  • A Hierarchical Latent Variable Encoder-Decoder model for Generating Dialogues

    WHY? Hierarchical recurrent encoder-decoder model(HRED) that aims to capture hierarchical structure of sequential data tends to fail because model is encouraged to capture only local structure and LSTM often has vanishing gradient effect. WHAT? Latent Variable Hierarchical Recurrent Encoder-Decoder(VHRED) tried to improve HRED by forcing to learn z with variational...


  • Forward-Backward Reinforcement Learning

    WHY? Reinforcement learning with sparse reward often suffer from finding rewards. WHAT? Forward-Backward Reinforcement Learning(FBRL) consists of forward and backward process. Forward process is like normal rl using memory to update Q function. In backward process, new model is introduced called backward model b. b is a neural network that...


  • A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning

    WHY? This paper wanted to catch non-linear dynamics of the object in video. WHAT? KVAE(Kalman Variational Autoencoder) combined Kalman filter with VAE to model dynamic latent variables. Linear Gaussian state space models are used to model Kalman filter and stable latent variables of the variational autoencoder. Matrices are the state...