[ContinualAI Reading Group] "Sequential mastery of multiple visual tasks: Networks naturally learn to learn and forget to forget"

[June 10th, 5 PM CEST], Guy Devison will present the paper:

Title: "Sequential mastery of multiple visual tasks: Networks naturally learn to learn and forget to forget"

Abstract: We explore the behavior of a standard convolutional neural net in a continual-learning setting that introduces visual classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks. This setting corresponds to that which human learners face as they acquire domain expertise serially, for example, as an individual studies a textbook. Through simulations involving sequences of ten related visual tasks, we find reason for optimism that nets will scale well as they advance from having a single skill to becoming multi-skill domain experts. We observe two key phenomena. First, \emph{forward facilitation}—the accelerated learning of task n+1 having learned n previous tasks—grows with n. Second, \emph{backward interference}—the forgetting of the n previous tasks when learning task n+1—diminishes with n. Amplifying forward facilitation is the goal of research on metalearning, and attenuating backward interference is the goal of research on catastrophic forgetting. We find that both of these goals are attained simply through broader exposure to a domain.

The speakers for this reading group will be:

• Guy Davidson

:round_pushpin: Youtube recording : https://youtu.be/nmCiDeh-lKY
:round_pushpin: Paper pre-print : https://arxiv.org/abs/1905.10837
:round_pushpin: Slides : https://drive.google.com/file/d/1kGhftcD1kZXUXRMAaR8SmXfNHnuoDaSF/view?usp=sharing

1 Like

Hello!

Thank you to all who attended, I enjoyed discussing this work with you.

The published version of the paper is available here: https://openaccess.thecvf.com/content_CVPR_2020/html/Davidson_Sequential_Mastery_of_Multiple_Visual_Tasks_Networks_Naturally_Learn_to_CVPR_2020_paper.html

The slides are available here: https://drive.google.com/file/d/1kGhftcD1kZXUXRMAaR8SmXfNHnuoDaSF/view?usp=sharing

Happy to chat about it further or answer any follow-up questions!

2 Likes

Thanks @guy.davidson and welcome to the ContinualAI forum! :smiley: