2024年9月5日 · MMMU is a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. …

2023年12月4日 · We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and …

2024年9月19日 · This is the organization page for all things related to MMMU, a Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI.

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI" - MMMU-Benchmark/MMMU

We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

2024年9月4日 · MMMU-Pro rigorously assesses multimodal models' true understanding and reasoning capabilities through a three-step process based on MMMU: (1) filtering out …

MMMU-Pro is an enhanced multimodal benchmark designed to rigorously assess the true understanding capabilities of advanced AI models across multiple modalities. It builds upon …

MMMU提出了四个挑战：1)全面性：跨越六大学科门类、30个大学学科的11.5K个大学水平问题;2)高度异构的图像类型;3)交织的文本和图像;4)基于深层学科知识的专家级感知和推理. 首先， …

2023年11月27日 · We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and …