Dynamic node group allocation
Provided are techniques for improving data locality for parallel applications running in a big data distributed file system with a dynamic node group. In response to a consumer job starting to read one or more files in a big data distributed file system having multiple nodes, node group information for the one or more files to be read is retrieved, wherein the node group information identifies nodes from the multiple nodes on which a producer job wrote the one or more files, and the consumer job is assigned to the nodes identified by the node group information to allow for local reading of the one or more files by the consumer job.
International Business Machines Corporation
G H H
G06F H04L H04L
inventional inventional inventional
Krishna K. Ron E. Yong Eric A. Xiaoyan
Bonagiri Liu Li Jacobson Pu