Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

[optimass]

Hey everyone!

Looking for feedback for our new paper (that is a little different from standard papers proposing a new model, hence the needed feedback):

Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

Online Fast Adaptation and Knowledge Accumulation: a New Approach to...

Here is a tiny summary:

We have made a lot of progress on catastrophic forgetting within the standard evaluation protocol,i.e. sequentially learning a stream of tasks and testing our models’ capacity to remember them all.
We think it’s time a new approach to CL, which is more aligned with real-life applications of CL.

Here are the main modifications we propose:

• bring CL closer to Online learning i.e. at test time, the model is continually learning and evaluated on its online predictions
• it’s fine to forget, as long as you can quickly remember (just like we humans do)
• we allow pretraining, (because you wouldn’t deploy an untrained CL system, right?) but at test time, the model will have to quickly learn new out-of-distribution tasks (because the world is full of surprise)
• the tasks distribution is actually a hidden Markov chain. This implies:
  • new and old tasks can re-occur (just like in real life). Better remember them quickly if you want to get a good total performance!
  • tasks have different lengths
  • and the tasks boundaries are unknown (task agnostic setting)

Bonus:
We provide a unifying framework to explain the space of machine learning setting {supervised learning, meta learning, continual learning, meta-continual learning, continual-meta learning} in case it was starting to get confusing

Finally:
The setting is not better than the others… We just think it should be studied as well

[vincenzo.lomonaco]

I’ll post here some of the answers to the thread on slack!

@vlomonaco: Hi @MassimoCaccia! this seems very interesting and aligned with our vision as well! I’ll give you feedback by the end of today!

@MassimoCaccia: thanks a lot Vincenzo!
I’m sure some ideas are not new, so if im missign some citations, please let me know!

@Martin_Mundt: Hey, the paper looks really great in terms of the direction it’s going. I am a big fan of moving away from the super constrained scenarios people have been practising up until now. I think I will have to read it again in order to give more concise feedback. But from a first read-through it seems that, while you are mentioning and somewhat analyzing out-of-distribution scenarios, you are pretty much missing any and all references and potential comparison.
We had explicitly looked at evaluation of continual learning in open world scenarios in our work: https://arxiv.org/pdf/1905.12019.pdf
If you take a look at all the references there in section 2.2 you can see that there exist a variety of other approaches. In particular I think that most existing probabilistic approaches (such as Bayesian approaches: variational continual learning, variational generative replay, our own paper that extends these) and approaches that are based on geometric properties have natural means to think outside “pure catastrophic forgetting in offline scenarios”. Most of these have things such as uncertainty estimation or data distribution approximation built in and thus are naturally more inclined towards online scenarios, open world learning etc. I think it would be really beneficial to your paper and the community in general, if you could go into more detail there, at least cite a larger body of prior work, or even better explicitly compare to some approaches that introduced the methods but didnt evaluate them in your precise context (instead of just not comparing to them because their initial evaluation was different/not appropriate to your context).
While it’s very important to bring new evaluation scenarios and other ideas into continual learning (such as OOD detection and meta-learning suitable evaluation), I think one should do more justice to all prior work and point out how one builds on top.
In your specific case, I believe this would be more references with respect to online learning and out-of-distribution detection in general, even if they are not specifically evaluated in some continual learning context. I would also try to craft some baselines based on these works to compare against in your specific experiments

@MassimoCaccia: feedback greatly appreciated! Gonna post a summary of my thoughts I sen you privately, so that people can follow if they want.

So, i’m sure that there is a gazillion methods that could work well in our setting because of e.g. OoD generalization mechanism. The goal of our paper is, however, simply to change the evaluation protocol such that better ideas (maybe like you mentioned) could prevail. Now, I don’t think I can adress all the papers that could work in our setting, right? Also, I’m not really proposing a new methodology and claiming SOTA. So it’s somewhat fine, no? However, if some of theses papers actually proposed a changed in the evaluation protocol (maybe like the one in open world scenario), not that I should address!

I agree that I should add more reference to Online Learning and OoD and bring the gaps a little better (the former was on my Todo list).

Thanks a lot Martin

@Dave_Rawlinson: “it’s fine to foget, as long as you can quickly remember (just like we humans do)” This is a very interesting angle I haven’t read elsewhere

@MassimoCaccia: I think that’s what you get when you mix Online Learning and Continual Learning. This part is highly inspired from https://arxiv.org/abs/1906.05201.

arXiv.org: Task Agnostic Continual Learning via Meta Learning

@vlomonaco: @MassimoCaccia congrats for the great paper, it looks very interesting!

A couple of questions:

1. How do you compute the accuracy? Is it a running average, one for each “task”?
2. What are roughly the training times and memory overheads of your strategy on the Tiered-ImageNet strategy?
3. Why did you not compare your strategy with other SOTA CL strategies?

@MassimoCaccia: Hey Vincenzo, thanks for reading!

1. it’s cumulative accuracy, or online or average. As you want to call it. The average is over all predictions
2. I’d have to ask, but they we’re surprisingly not that long on a Single GPU
3. BGD seemed like a good Task-Agnostic CL baseline. Then MetaBGD is better in our setting, because it can adapt more quickly. I’m not sure how to standard CL methods would apply in the task-agnostic setting (so class incremental algorithms don’t really work out of the box) any suggestion?

@vlomonaco: great!

All the methods that we list in these two papers below, can be used in a task-agnostic setting!

https://arxiv.org/abs/1907.03799
https://arxiv.org/pdf/1806.08568.pdf

and also the one @Chris_Kanan has just posted:
https://arxiv.org/abs/1910.02509

arXiv.org: Fine-Grained Continual Learning

arXiv.org: REMIND Your Neural Network to Prevent Catastrophic Forgetting

@MassimoCaccia: So, IMO task-agnostic means that you can’t trivially infer the task from Y. So in classification, e.g., the class “dog” can could be represented by the first output dimension in a task, and the second in another (just like in meta-learning). Can these method handle this. IMO, incremental class learning, by construction, is task aware. But, I could be convinced otherwise…

@vlomonaco: Most class-incremental learning techniques don’t have the notion of tasks at all, so you can’t infer the “task” from Y. Every X encountered at every moment can be classified as any Y encountered so far. We explain this well in Sec 1.2 of this paper: https://arxiv.org/pdf/1806.08568.pdf

The only limitation of the techniques referred above is that you cannot have different inferred classes Y for the same sample X. That’s because there’s no task notion and you need an additional T signal to disambiguate.

I believe this is happening in your settings only for the “Synbol” benchmark where you use different outputs for the same input data for different tasks, right?

@MassimoCaccia: Yes, that was one goal of the Synbols dataset. i’ll check Sec 1.2 and get back to you

[optimass]

Ok, so I looked at section 1.2 of your paper https://arxiv.org/pdf/1806.08568.pdf. In my vocabulary, this is not a task-agnostic setting. I think we just have different solutions. For me, if you are doing incremental class learning, you are in the task-aware setting by construction.

I’ll try to reword my definition: task-agnostic for me is the scenario where the labels might not always map to the same output. It is more general. (I know it’s weird in classification… However, all of meta-learning is that way).

So don’t think I could apply the methods you are proposing in this setting.

[vincenzo.lomonaco]

I understand what you mean now, thanks.

You can easily apply them once you infer the task you are solving using separated “heads”!

The fact is that, once you infer the task, the problem becomes easier, since you can separate the output space. This is why we focus on what we call the “Single-Incremental-Task” setting, where you can’t separate it by definition and it has been proved to be harder than the Multi-Task setting.

Then of course you can make it more complicated, adding Multiple-Incremental-Tasks (different mappings f(x) for the same x you learn continually), but we believe that’s not at the center of the CL problem.

In the end, what we want to solve in CL is pretty simple. Learn:

f: X -> Y

Where X and Y can change over time. When you add the notion of task, you’re just adding complexity but the core of the issue is the same:

f: X, T -> Y

Where X, T and Y can change over time.

This is just my opinion though!

[optimass]

I agree that my setting is “easier” because you can reduce the output space if you correctly infer the task from the context.

I don’t see why it’s not central to CL though. In the end, I think that a lot of applications outside image classification can benefit from modeling the context. E.g. time series forecasting, autonomous driving, robotics, recommender systems, partially observable games (i’d be happy to dig in if it’s not super clear why).

Even for image classification, I could see how you would leverage context. E.g. someone using an Animal Classifier app in the jungle: if the app is confident that it is in the jungle, it could switch to using its “jungle” output head, for better accuracy.

So, IMO, we should study both settings, without one being more central than the other. Just depends on the set of applications you want to make progress on

Hope we can continue the discussion. I think we are making progress

[vincenzo.lomonaco]

Absolutely! Every setting has the right to exist and to be studied, especially when targeting real-world application! Moreover, a good strategy should be able to work in any setting

What I meant is that the core objective of CL today is to learn robust, high-level features from a stream of data drawn from ever-changing data distributions and this can be studied independently by the presence of multiple tasks.

So, in general, I prefer to focus on settings with just one task to remove the “noise” introduced by different settings that can artificially / indirectly help to solve the scenario without providing roboust and scalable solutions to learn continually, but this is just my approach!