Contrastive studying usually matches pairs of associated views amongst a lot of unrelated unfavourable views.
Views could be generated (e.g. by augmentations) or be noticed. We examine matching when there are greater than two associated views which we name poly-view duties, and derive new illustration studying aims utilizing data maximization and enough statistics.
We present that with limitless computation, one ought to maximize the variety of associated views, and with a hard and fast compute funds, it’s useful to lower the variety of distinctive samples while growing the variety of views of these samples.
Specifically, poly-view contrastive fashions educated for 128 epochs with batch dimension 256 outperform SimCLR educated for 1024 epochs at batch dimension 4096 on ImageNet1k, difficult the assumption that contrastive fashions require giant batch sizes and plenty of coaching epochs.
![](https://mlr.cdn-apple.com/media/imagenet1k_flops_de2d1ab0c8.png)