An Investigation of the Robustness of Perception Based Control

Modern day control systems such as autonomous driving typically utilize image and video signals to produce feedback control. However, adversarial noise can often interfere with label prediction and thus lead to decreased robustness. This project aims to investigate adversarial robustness of convolutional neural network (CNN) based machine perception for identifying the position of a driving car. Certain area of machine perception for control are robust to some objects but may be fragile towards others. Therefore, we tested which type of noise affected the model the most and retrained the model with the adversarial noise in mind.

​

We generated a dataset containing 2,000 images with 50 different calculated trajectories for a car's position and 40 base images containing a car on a road. Then, we built and trained our convolutional neural network with four different layers in order to obtain 52,673 trainable parameters.