• REEF: Retainable Evaluator Execution Framework

    Summary Resource managers successfully support distributed computing framework by allocating jobs to nodes in a cluster. Though this RMs are good at utilizing the resources in a cluster, the process of implementation remains quite bothersome dealing the same tasks repetitively for every RMs and applications. Retainable Evaluator Execution Framework(REEF) is...


  • Apache Hadoop YARN: Yet Another Resource Negotiator

    Summary Apache Hadoop is widely used not only for MapReduce jobs but also any kinds of data processing. The first version of Hadoop had 3 critical drawbacks. First, the framework is hard-coded to MapReduce only. Second, the slots for map and reduce functions are static limiting utilization. Third, JobTracker managed...


  • MapReduce: Simplified Data Processing on Large Clusters

    Summary MapReduce is a programming model that parallelize the computation on large data. The computation consists of two functions: a map and a reduce function. First, input data are partitioned into M splits. Map function takes a partition of inputs and produce a set of intermediate key/value pairs. By partitioning...


  • Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center

    Summary Mesos is a platform that provides cross-framework resource allocation. The greatest challenge for cross-framework resource sharing is that various kinds of frameworks may have their own different scheduling. Mesos overcome this problem by adopting two-level scheduling mechanism called resource offers. Framework of Mesos Resource offer process Mesos API function...


  • Relational Recurrent Neural Network

    WHY? While conventional deep learning models have performed well on inference regarding individual entities, few models have been proposed focusing on inference regarding the relations among entities. Relational network aimed to capture relational information from images. In this paper, relational recurrent neural network tried to improve former memory augmented neural...