2024年3月23日 · To address this issue, we introduce VLM-CPL, a novel approach based on consensus pseudo labels that integrates two noisy label filtering techniques with a semi-supervised learning strategy. Specifically, we first obtain prompt-based pseudo labels with uncertainty estimation by zero-shot inference with the VLM using multiple augmented views of ...

2024年7月11日 · Vision-language models (VLMs) can learn high-quality representations from a large-scale training dataset of image-text pairs. Prompt learning is a popular approach to fine-tuning VLM to adapt them to downstream tasks. Despite the satisfying performance, a major limitation of prompt learning is the demand for labelled data. In real-world scenarios, we may only obtain candidate labels (where the ...

2024年3月8日 · In the field of Class Incremental Object Detection (CIOD), creating models that can continuously learn like humans is a major challenge. Pseudo-labeling methods, although initially powerful, struggle with multi-scenario incremental learning due to their tendency to forget past knowledge. To overcome this, we introduce a new approach called Vision-Language Model assisted Pseudo-Labeling (VLM-PL ...

Label Propagation：一种基于图的 标签传播 方法，能够在标签有限的情况下进行有效的学习，但在处理大规模数据集时计算复杂度高。 研究方法 这篇论文提出了一种基于图的方法，用于VLM的零样本或少样本适应和推理。

2024年11月26日 · This is the repository of Vision Language Models for Vision Tasks: a Survey, a systematic survey of VLM studies in various visual recognition tasks including image classification, object detection, semantic segmentation, etc.For details, please refer to: Vision-Language Models for Vision Tasks: A Survey []. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024

2024年3月27日 · 为了应对这一挑战，我们引入了VLM-CPL（Consensus Pseudo Labels from Vision-Language Models），一种基于共识伪标签的新方法，该方法结合了两种噪声标签过滤技术，并采用了半监督学习策略。具体来说，我们首先通过VLM的零样本推理能力，使用输入的多个增强视图获得基于 ...

We design a novel scheme for the OOD score collaborated with negative labels. Theoretical analysis helps to understand the mechanism of negative labels. Extensive experiments demonstrate that our method NegLabel achieves state-of-the-art performance on various OOD detection benchmarks and generalizes well on multiple VLM architectures.

The commands in run.sh will by default load the cached preference labels; you can use cached_label_path=None to not use the cached labels and query the VLM online during training. If you wish to fully reproduce the results in the paper, please train without using the provided cached labels, and generate the VLM preference labels online using ...

To address this issue, we introduce VLM-CPL, a novel approach based on consensus pseudo labels that integrates two noisy label filtering techniques with a semi-supervised learning strategy. Specifically, we first obtain prompt-based pseudo labels with uncertainty estimation by zero-shot inference with the VLM using multiple augmented views of ...

OLA-VLM 仅在 inference 期间使用单个 visual encoder 就实现了这些结果，使其比多编码器系统效率高得多。 为了进一步验证其有效性，研究人员分析了 OLA-VLM 学习到的表示。 Probing experiments 表明，该模型在其中间层实现了卓越的视觉特征对齐。这种对齐显著增强了模型 ...