Summary

Resource managers successfully support distributed computing framework by allocating jobs to nodes in a cluster. Though this RMs are good at utilizing the resources in a cluster, the process of implementation remains quite bothersome dealing the same tasks repetitively for every RMs and applications. Retainable Evaluator Execution Framework(REEF) is a meta framework between RMs and applications which not only marginalize repetitive trivial works but also enable container and state reuse across different frameworks. REEF contains few key concepts. Driver allocates resources and schedule tasks. Evaluator is a runtime environment that executes tasks. Context is a state management environment in a evaluator that that retain state. Servie is a library in a evaluator that contains frequently used functionalities. REEF provides higher abstraction on top of RM framework letting developers bypassing trivial implementations.

Strengths

Super convinient for devlopers to use atop various kinds of frameworks.

Weaknesses

REEF runtime infrastructure may have to be updated for every modifications or updates of one of many frameworks.

New things I learned

I didn’t know that the state of a container can be retained.

Discussion topics

When would REEF be especially useful? Would REEF be still useful under static framework use?

Chun, Byung-Gon, et al. “Reef: Retainable evaluator execution framework.” Proceedings of the VLDB Endowment 6.12 (2013): 1370-1373.