Academy & Industry Research Collaboration Center (AIRCC)

Volume 11, Number 13, August 2021

Research on Task Scheduling Strategy based on the Trustworthiness of MapReduce

  Authors

QIN Jun1, 2, SONG Yanyan3 and ZONG Ping4, 5, 1Communication University of China, Nanjing, China, 2Nanjing University of Posts and Telecommunications, China, 3Communication University of China, Nanjing, China, 4Nanjing University of Science and Technology Zijin College, China, 5Nanjing University of Posts and Telecommunications, China

  Abstract

With the rapid development and popularization of information technology, cloud computing technology provides a good environment for solving massive data processing. Hadoop is an open-source implementation of MapReduce and has the ability to process large amounts of data. Aiming at the shortcomings of the fault-tolerant technology in the MapReduce programming model, this paper proposes a reliability task scheduling strategy that introduces a failure recovery mechanism, evaluates the trustworthiness of resource nodes in the cloud environment, establishes a trustworthiness model, and avoids task allocation to low reliability node, causing the task to be re-executed, wasting time and resources. Finally, the simulation platform CloudSim verifies the validity and stability of the task scheduling algorithm and scheduling model proposed in this paper.

  Keywords

Cloud Environment, Failure Recovery Mechanism, Task Scheduling Algorithm.