View in #questions-forum on Slack
@Hitesh_Ulhas_Vaidya: Hello !channel ,
Could someone please let me know how data samples are generally labeled for continual learning problem? For example, if I am using Split-MNIST in an incremental fashion, how are they labeled in continual learning?
@andcos: Hi @Hitesh_Ulhas_Vaidya, in Split MNIST labels are the same as MNIST. This applies to class incremental settings in which you have a single output layer. If you use separate output layers (heads) with 2 output units for each pair of digits, labels are 0,1 only.
@Hitesh_Ulhas_Vaidya: so, if the model receives classes incrementally and the output layer has just one node, the current class will be labeled as 1 and the rest as 0?
also, what happens in case of replay then?
@andcos: if you use neural networks you usually have output layers with more than one node because the model has to distinguish at least between two classes in order to learn something meaningful. Anyway, if you have only one node you will have all labels with value 0. In case of replay, each replay pattern is associated to its original label so that you will reuse the same output units used when you trained on that pattern.