Volume 10, Number 5

Test-cost-sensitive Convolutional Neural Networks with Expert Branches

  Authors

Mahdi Naghibi¹, Reza Anvari¹, Ali Forghani¹ and Behrouz Minaei², ¹Malek-Ashtar University of Technology, Iran and ²Iran University of Science and Technology, Iran

  Abstract

It has been proven than deeper convolutional neural networks (CNN) can result in better accuracy in many problems, but this accuracy comes with a high computational cost. Also, input instances have not the same difficulty. As a solution for accuracy vs. computational cost dilemma, we introduce a new test-cost-sensitive method for convolutional neural networks. This method trains a CNN with a set of auxiliary outputs and expert branches in some middle layers of the network. The expert branches decide to use a shallower part of the network or going deeper to the end, based on the difficulty of input instance. The expert branches learn to determine: is the current network prediction is wrong and if the given instance passed to deeper layers of the network it will generate right output; If not, then the expert branches stop the computation process. The experimental results on standard dataset CIFAR-10 show that the proposed method can train models with lower test-cost and competitive accuracy in comparison with basic models.

  Keywords

Test-Cost-Sensitive Learning, Deep Learning, CNN withExpert Branches, Instance-Based Cost