A Metamodel-Based General-Purpose Autocalibration Tool for Simulation Models.

Background. Simulation calibration is the process of configuring a simulation model's parameters to improve the agreement between the model output and the desired calibration targets (e.g., observed historical data). For most realistic simulation models, this calibration process can be quite computationally expensive, as it requires running the simulation model for each parameter combination. To alleviate this problem, metamodels offer a tradeoff between accuracy and computational efficiency for extensive simulative analysis with a highly complex parameter space. Method. In this study, we examine 4 simulation calibration approaches. Randomly-Simulate (RS) is a simulation-based benchmark widely used in the literature. Optimally-Predict (OP) is an optimization-based approach that we adapt to the simulation calibration setting. Building on these 2 baselines, we introduce 2 hybrid strategies, Predict-then-Simulate (PtS) and Simulate-then-Predict (StP), which combine simulation runs and metamodel-based optimization in complementary orders. We compare all 4 methods in terms of calibration accuracy and computational cost. Results. While the metamodel-based OP approach substantially reduced computational cost relative to RS and identified parameter combinations near the optimal configuration, the hybrid strategies delivered superior calibration performance. In particular, the PtS approach, which combines metamodel-based optimization with targeted simulation refinement, achieved on average a 46% reduction in total actual error compared with the RS benchmark, while maintaining computational efficiency. Conclusions. The study introduces a novel metamodel-based optimization approach to simulation calibration and illustrates its potential benefits for computationally expensive studies. While developed for deterministic targets, the method provides a foundation for future extensions to settings involving stochastic simulation outputs and other forms of model uncertainty. An open-access Python implementation of the proposed framework is provided to facilitate adoption and reproducibility.HighlightsA metamodel-based optimization approach is proposed for calibrating simulation model parameters, which can offer computational advantages particularly in cases in which direct calibration is expensive due to complex or high-dimensional simulation models.A hybrid approach that narrows down the search space via metamodel-based optimization and then uses simulation runs for fine-tuning offers a sweet spot in the tradeoff between computational efficiency and accuracy.