2024年9月5日 · MMMU is a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

Our MMMU benchmark introduces key challenges to multimodal foundation models, as detailed in a figure. Among these, we particularly highlight the challenge stemming from the requirement for both expert-level visual perceptual abilities and deliberate reasoning with …

2023年12月4日 · We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

2024年9月19日 · This is the organization page for all things related to MMMU, a Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI.

2024年12月31日 · No, ITT just went up Dec 30 (5am, GMT+8). Before I was in MMMU Mocha last week while there was an ongoing maintenance in the Arcade. During that period I was EQd in Mocha despite passing the assessment.

2024年9月4日 · MMMU-Pro provides a more rigorous evaluation tool, closely mimicking real-world scenarios and offering valuable directions for future research in multimodal AI. Subjects: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV)

We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI" - MMMU-Benchmark/MMMU

2023年11月27日 · We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.

2024年5月30日 · The MMMU Benchmark (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark) is a new benchmark for evaluating multimodal models on massive multi-discipline...