[ContinualAI Reading Group] Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams

[02-10-2020] This Friday 5pm CEST, for the ContinualAI Reading Group, Matthias De Lange will present the paper:

Title: "Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams"

Abstract: Attaining prototypical features to represent class distributions is well established in representation learning. However, learning prototypes online from streams of data proves a challenging endeavor as they rapidly become outdated, caused by an ever-changing parameter space in the learning process. Additionally, continual learning does not assume the data stream to be stationary, typically resulting in catastrophic forgetting of previous knowledge. As a first, we introduce a system addressing both problems, where prototypes evolve continually in a shared latent space, enabling learning and prediction at any point in time. In contrast to the major body of work in continual learning, data streams are processed in an online fashion, without additional task-information, and an efficient memory scheme provides robustness to imbalanced data streams. Besides nearest neighbor based prediction, learning is facilitated by a novel objective function, encouraging cluster density about the class prototype and increased inter-class variance. Furthermore, the latent space quality is elevated by pseudo-prototypes in each batch, constituted by replay of exemplars from memory. We generalize the existing paradigms in continual learning to incorporate data incremental learning from data streams by formalizing a two-agent learner-evaluator framework, and obtain state-of-the-art performance by a significant margin on eight benchmarks, including three highly imbalanced data streams.

The event will be moderated by: Vincenzo Lomonaco

:round_pushpin: Eventbrite event (to save it in you calendar and get reminders): https://www.eventbrite.com/e/continual-prototype-evolution-learning-online-from-non-stationary-data-tickets-125366585833
:round_pushpin: Google Meet Link: https://meet.google.com/sof-dbec-tpi

1 Like

Thank you for having me and for your interesting questions! You can find the slides on my website: https://mattdl.github.io/extra/2020_CoPE/20201011_CoPE_readinggroup_continualAI.pdf
For any questions, feel free to contact me, Iā€™d be happy to help :slight_smile: