Reload: Selective Data Forgetting and Selective Data Replacement
Date:
This talk covered the work I performed over the summer of 2024 at the Vector Institute as a research intern under Dr. Rahul Krishnan.
Machine Unlearning is an important task in machine learning surrounding making a model selectively ‘forget’ part of its training data. In some cases, the data may be unavailable, and conventional methods require access to unlearn. Those that do not, do not perform well. In this talk I introduce Reload, a new method I developed with Dr. Rahul Krishnan which is inspired by memorization in neural networks and gradient-based input saliency maps to retrain part of the network. I justify the need for Reload, and the theoretical motivations and justifications for its design, as well as demonstrate its state-of-the-art performance on unlearning tasks. I also introduce an extension of the unlearning problem to selective data replacement, to correct or update part of the training data. I define the problem, empirical evaluation tasks, justify the use of Reload for this task, and demonstrate its performance on replacing part of the training set. My presentation wraps up with a discussion of my next steps to complete this project.