Categorization

Operating system

Machine learning

Trustworthiness

Artificial intelligence

Exploit

Data science

Backdoor

Computer science

Malware

Process (computing)

Computer security

IEEE Transactions on Pattern Analysis and Machine Intelligence

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space.

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses