WIQOR: A dataset for what-if analysis of Operations Research problems

We formalize the mathematical program modification (MPM) task, in which the goal is to revise a mathematical program according to an inquiry expressed in natural language. These inquiries, which we refer to as what-if questions, express a desire to understand how the optimal solution to an optimization problem changes with the addition, deletion or revision of constraints. In detail, each MPM instance is a triple consisting of: 1) a natural language specification that summarizes an optimization problem, 2) the canonical formulation of the problem, and 3) a natural language "what-if" question. The goal is to predict the updated canonical formulation with respect to the question. To support the study of this task, we construct WIQOR, a dataset of 1,946 MPM instances, derived from NL4OPT (Ramamonjison et al., 2023), but with the number of decision variables extended to more than 30 for some problems. In experiments, we observe that Llama 3.1 70B instruct under the in-context learning paradigm achieves 69% accuracy on the easiest test instances, but only 36% accuracy on the most complicated problems. We release WIQOR in the hopes of spurring additional study of MPM and ultimately enabling non-technical users to conduct what-if analyses without the help of technical experts.