what is a favorable replay pattern for a standard replay algorithm for streaming data containing multiple tasks sequentially while the number of data points for a task is not defined?
I could imagine having a memory of size m for each previous task. New data points belonging to a new task from the data stream are collected until a batch of fixed size is reached. Then this new task data batch is concat with replay samples for previous tasks from the memory. This merged batch is used to train the NN.
But how do I address the problem of unknown length of the data stream belonging to the new task? Usually multiple epochs of an entire data set are used to train a network, but since the data stream could be infinitely long, the mini-batch-wise training process could theoretically last forever. Further all the data points are shown only once to the network. Is it useful to store some data of the current task and replay them together with data from previous tasks from the memory and the newest read data from the data stream?
Is there a better way to deal with this problem?