Academy & Industry Research Collaboration Center (AIRCC)

Volume 10, Number 06, June 2020

Systematic Attack Surface Reduction for Deployed Sentiment Analysis Models


Josh Kalin1, David Noever2 and Gerry Dozier1, 1Auburn University, USA and 2PeopleTec, Inc, USA


This work proposes a structured approach to baselining a model, identifying attack vectors, and securing the machine learning models after deployment. This method for securing each model post deployment is called the BAD (Build, Attack, and Defend) Architecture. Two implementations of the BAD architecture are evaluated to quantify the adversarial life cycle for a black box Sentiment Analysis system. As a challenging diagnostic, the Jigsaw Toxic Bias dataset is selected as the baseline in our performance tool. Each implementation of the architecture will build a baseline performance report, attack a common weakness, and defend the incoming attack. As an important note: each attack surface demonstrated in this work is detectable and preventable. The goal is to demonstrate a viable methodology for securing a machine learning model in a production setting.


Machine Learning, Sentiment Analysis, Adversarial Attacks, Substitution Attacks.