Volume 9, Number 7, June 2019

A Machine Learning Algorithm in Automated Text Categorization of Legacy Archives

  Authors

¹Dali Wang, ²Ying Bai and ¹David Hamblin, ¹Christopher Newport University, USA and ²Johnson C. Smith University, USA

  Abstract

The goal of this research is to develop an algorithm to automatically retrieve critical information from raw data files in NASA’s airborne measurement data archive. The product has to meet specific metrics in term of accuracy, robustness and usability, as the initial decision-tree based development has shown limited applicability due to its resource intensive characteristics. We have developed an innovative solution that is much less resource intensive while offering comparable performance. As with many practical applications, the data available are noisy and correlated; and there is a wide range of features that are associated with the information to be retrieved. The proposed algorithm uses a decision tree to select features and determine their weights. A weighted Naive Bayes is used due to the presence of highly correlated inputs. The development has been successfully deployed in an industrial scale, and the results show that the development is well-balanced in term of performance and resource requirements.

  Keywords

Machine Learning, Classification, Naïve Bayes, Decision Tree

Conference Proceedings