MaD Physics
Evaluating Information Seeking Under Constraints in Physical Environments

Abstract

Scientific discovery is fundamentally a resource-constrained process that requires navigating complex trade-offs between the quality and quantity of measurements due to physical and cost constraints. Existing benchmarks for evaluating agents for scientific discovery focus on either static knowledge-based reasoning or unconstrained experimental design tasks, and do not capture the ability to make measurements and plan under constraints.

To bridge this gap, we propose Measuring and Discovering Physics (MaD Physics), a benchmark to evaluate the ability of agents to make informative measurements and conclusions subject to constraints on the quality and quantity of measurements. The benchmark consists of three environments, each based on a distinct physical law. To mitigate contamination from existing knowledge, MaD Physics includes altered physical laws. In each trial, the agent makes measurements of the system until it exhausts an allotted budget, and then must infer the underlying physical law to make predictions about the future state of the system.

MaD Physics evaluates two fundamental capabilities of scientific agents: inferring models from data and planning under constraints. We also demonstrate how MaD Physics can be used to evaluate other capabilities such as multimodality and in-context learning. We benchmark agents on MaD Physics using four Gemini models (2.5 Flash Lite, 2.5 Flash, 2.5 Pro, and 3 Flash), identifying shortcomings in their structured exploration and data collection capabilities, and highlighting directions to improve their scientific reasoning.

Environments

Main Results

We benchmark a minimal agent scaffold with code execution across Gemini 2.5 Flash Lite, 2.5 Flash, 2.5 Pro, and 3 Flash. Performance generally improves with model capability, and a Strategy system prompt inspired by Bayesian experimental design tends to help. However, even the strongest models struggle to recover correct symbolic forms of the underlying laws, and smaller Gemini 2.5 models often produce out-of-bounds predictions in classical mechanics, indicating headroom for both better scaffolds and base capabilities.

Predictions are not clipped — entries far above 1 in the Classical table reflect runaway out-of-bounds predictions, separating models that produce stable predictions from those that don't.

Variants

Beyond predictive error on the default environments, MaD Physics supports several variants that probe additional capabilities: image-based observations, in-context learning across episodes, and parameter inference under a known structural form.

Visual observations

An additional variant of the Classical environment provides only image renderings of the system instead of numerical state values. Trends across model capability and altered laws hold, but errors are noticeably larger.

Visual Classical Mechanics

Model	Prompt	Pred. Err. ↓
		Normal	κ=10
Gemini 2.5 Flash	Base	6.81	1786.13
	+ Strat.	3.16	15.64
Gemini 2.5 Pro	Base	4.56	21.65
	+ Strat.	0.65	13.23

In-context learning

Prediction error on Classical Mechanics across episodes for Gemini 2.5 Pro and Gemini 3 Flash. Gemini 3 Flash starts lower and continues to improve, while 2.5 Pro fails to learn under altered physics.

Parameter inference

Active-sensing variant: estimate κ (the inertial-memory coupling) given the model’s structural form. Gemini 2.5 Pro consistently underestimates κ, indicating a bias toward standard physics.

BibTeX

@article{jain2026madphysics,
  title         = {{MaD} Physics: Evaluating Information Seeking Under Constraints in Physical Environments},
  author        = {Jain, Moksh and Bennani, Mehdi and Bausch, Johannes and
                   Chervonyi, Yuri and Georgiev, Bogdan and Osindero, Simon and
                   Toma\v{s}ev, Nenad},
  year          = {2026},
  eprint        = {2605.10820},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2605.10820}
}