Stability and Significance Analysis of Daily Food Price Forecasting Models

Abstract

Food price forecasting has been widely studied using machine learning, yet most published work reports accuracy from a single data split without a baseline comparison or a significance test. This study evaluates the stability and statistical significance of such comparisons. Eleven models comprising four baselines, two statistical models, and five machine learning algorithms were tested on eight strategic food commodities in East Java Province using 2,380 daily observations from April 2017 to September 2026 obtained from the Strategic Food Price Information Centre of Bank Indonesia. Evaluation employed six-fold expanding window validation with a thirty-step horizon, using a scaled error measure and a skill score relative to a naive benchmark as primary metrics, followed by Friedman and Nemenyi testing. No model proved significantly superior to the naive benchmark, which itself ranked third among the eleven models. Rankings proved highly unstable: the standard deviation across folds reached 35 to 106 percent of mean values, the best model changed for half of the commodities after cleaning only 0.1 percent of observations, and percentage error correlated negatively with the skill score. These findings indicate that claims of model superiority require baseline comparison, multi-fold validation, and significance testing.