Volume 15, Number 6

Trust Metric-Based Anomaly Detection via Deep Deterministic Policy Gradient Reinforcement Learning Framework

  Authors

Shruthi N¹ and Siddesh G K², ¹Visvesvaraya Technological University, India, ²ALVA’s Institute of Engineering & Technology, India

  Abstract

Addressing real-time network security issues is paramount due to the rapidly expanding IoT jargon. The erratic rise in usage of inadequately secured IoT- based sensory devices like wearables of mobile users, autonomous vehicles, smartphones and appliances by a larger user community is fuelling the need for a trustable, super-performant security framework. An efficient anomaly detection system would aim to address the anomaly detection problem by devising a competent attack detection model. This paper delves into the Deep Deterministic Policy Gradient (DDPG) approach, a promising Reinforcement Learning platform to combat noisy sensor samples which are instigated by alarming network attacks. The authors propose an enhanced DDPG approach based on trust metrics and belief networks, referred to as Deep Deterministic Policy Gradient Belief Network (DDPG-BN). This deep-learning-based approach is projected as an algorithm to provide “Deep-Defense” to the plethora of network attacks. Confidence interval is chosen as the trust metric to decide on the termination of sensor sample collection. Once an enlisted attack is detected, the collection of samples from the particular sensor will automatically cease. The evaluations and results of the experiments highlight a better detection accuracy of 98.37% compared to its counterpart conventional DDPG implementation of 97.46%. The paper also covers the work based on a contemporary Deep Reinforcement Learning (DRL) algorithm, the Actor Critic (AC). The proposed deep learning binary classification model is validated using the NSL-KDD dataset and the performance is compared to a few deep learning implementations as well.

  Keywords

Deep Deterministic Policy Gradient, Reinforcement Learning, Security, Anomaly Detection, Confidence Interval, LSTM, Actor Critic.