View in #questions-forum on Slack
@vlomonaco: !channel I need your help here! I would like to maintain a thread with all the CL algorithms tested on ImageNet! Can you post the link to the papers you know have been tested on ImageNet (no mini-Imagenet) in this thread? Thanks!
@Chris_Kanan: Great idea, but some key caveats when comparing ImageNet results across papers!
- Base initialization varies across papers a lot. Typically between 100 (10%) - 500 (50%) of classes.
- For incremental batch learning papers, batch size matters a lot. Many of them train with batches of 100 classes, which means the incremental batch contains ~100K images.
- The paradigm matters too, where online / streaming learning is pretty rare (just my group, I think) with most others doing big batches.
Basically, without keeping these details identical, you can’t easily compare results across papers.
My PhD student @Tyler_L._Hayes recently compiled an updated list, which she can probably share for ImageNet results.
@vlomonaco: Totally agree Chris! For now let’s just make a list, then I can try to organize them better!
@Tyler_L._Hayes: Hey @vlomonaco, here is a list of the papers I know that have tested on ImageNet-1K. I’ll update it if I forgot any others
iCaRL - CVPR-2017 - https://arxiv.org/abs/1611.07725
EEIL - ECCV-2018 - https://arxiv.org/abs/1807.09536
Unified Classifier - CVPR-2019 - http://openaccess.thecvf.com/content_CVPR_2019/papers/Hou_Learning_a_Unified_Classifier_Incrementally_via_Rebalancing_CVPR_2019_paper.pdf
BiC - CVPR-2019 - https://arxiv.org/abs/1905.13260
IL2M - ICCV-2019 - http://openaccess.thecvf.com/content_ICCV_2019/html/Belouadah_IL2M_Class_Incremental_Learning_With_Dual_Memory_ICCV_2019_paper.html
ScaIL - WACV-2020 - https://arxiv.org/abs/2001.05755
Deep SLDA - arXiv-2020 - https://arxiv.org/abs/1909.01520
REMIND - arXiv-2020 - https://arxiv.org/abs/1910.02509
@Chris_Kanan: I don’t think ImageNet is ideal, since videos seem more natural. We really need something like a CORe10000. But ImageNet is a really good baseline until we have something better
@vlomonaco: Thank you so much @Tyler_L._Hayes! and yes @Chris_Kanan, in our lab we think the same!
For the future we are thinking on merging iCub-Transformation, CORe50 and OpenLORIS or to use YouTube-BoundingBoxes. That would be great!
P.s. we may release an extended version of CORe50 very soon!