Hi everybody! I have some questions regarding task-free continual learning. I have just started digging into CL, so I apologize in advance if my questions do not make sense.
I was taking a look at [1] and I agree with the authors that a true continual learning agent should learn online and should not rely on the knowledge of task labels and boundaries. Besides that, IMHO one of the strengths of [1] is that they do experiments on weak and self-supervision. In fact, it seems that in this context the ability of learning without supervision is crucial for three reasons:
- humans are very slow at labeling, so most of the advantages of online learning become useless if the system needs to wait for the label
- I can’t think of a reason why the task label should not be known if the data has been manually annotated. Couldn’t the annotator just give the task label as well?
- correct me if I’m wrong, but knowing the class labels entails knowledge of the task label by definition since the task label represents a subset of the set of classes. Also, it is very easy to just use the class label to detect a task change (in python
if current_label not in already_seen_labels: new_task())
note that point 3 is probably related to this sentence I found in section 3 of [2]:
However, it is only applied to supervised tasks, and can exploit the “label” trick, inferring the task based on the class label
Given these thoughts, the task-free setting seems unreasonable under the supervised paradigm. Nonetheless, many recent works (for example [3][4] but I am confident that there are others that I cannot find now) seem to go in that direction. At first, I thought that it might be that supervised learning is used as a toy problem for testing task-free continual learning, but then I found out that these methods explicitly rely on labels (e.g. for picking data points to store in the buffer) and therefore cannot be trivially adapted for unsupervised learning.
DISCLAIMER: I assume there is something wrong with my understanding since these authors are super experts in the field. What am I missing here?
Furthermore, I am confused about the unsupervised task-free continual learning setting presented in [2]. They only include CL experiments where a single class is presented to the model at a time, so this would be something like unsupervised task-free class-incremental continual learning. If this is true it means that the class label needs to be known at training time, otherwise, how do you split the data? Also, it can be that you suppose that the data for each class come from different “sources” and therefore you don’t need to label it, but then you can just use the “source” as a label. Hence it seems to me that this unsupervised CL is not really unsupervised. Again, I am probably wrong, I just want to understand 
Moreover, it is true that in [2] there are experiments on
continuous drift, similar to the sequential case, but with classes gradually introduced by slowly increasing the number of samples from the new class within a batch
but my understanding is that this still requires the knowledge of the class label (which btw in this case is the same as the task label).
Also, because of how the dynamic expansion is devised, I reckon that it is not possible for CURL to handle more than one new class at a time, am I right?
Can’t wait for your replies. Sorry for the wall of text.
[1] Task-Free Continual Learning
[2] Continual Unsupervised Representation Learning
[3] Online Continual Learning with Maximal Interfered Retrieval
[4] Gradient based sample selection for online continual learning
. Also, I think it is unreasonable to expect that only one class will come at you at a time (class-incremental). Probably, as 
