Few-Shot Learning of Fine-Grained Concepts

Abstract

The talk addressed the topic of recognizing fine-grained visual concepts in the extremely low-data regime of few-shot learning. This is a very challenging problem (that, however, often occurs in practice), where the concept that needs to be recognized is orthogonal to the more coarse-grained concepts that neural networks are typically able to recognize when pre-trained on datasets such as ImageNet, but even on much larger datasets, such as the 400 million image-text pairs used to train CLIP. Consequently, there is a challenge in both: (i) how to adequately demonstrate such orthogonal concept through a support set of a few-shot task, and (ii) how to perform pretraining and construct a few-shot classifier so as to be able to extract features relevant to the target few-shot task. The talk will outline several approaches to the problem and also consider how current advances in multi-modal learning could help to make these kinds of tasks easier to solve in the future.