BBQ Weather Prediction in Basel Using Ensemble Machine Learning

Abstract

Weather-dependent decision making, such as planning an outdoor barbecue (BBQ), benefits from short-term forecasts that are both accurate and honestly evaluated. This study addresses two overlooked risks in applied weather classification: label leakage from same-day rule-based targets, and validation-set overfitting caused by repeated model-selection decisions. Using the ECA&D Basel daily weather records (2000–2010), the original BBQ-weather label was found to be fully determined by same-day precipitation, so the task was reframed as next-day forecasting through one-day lag features and target shifting. Data were split chronologically into training, validation, and test sets (60:20:20) to preserve temporal independence. Five heterogeneous classifiers (CatBoost, LightGBM, RUSBoost, Nearest Centroid, SGDClassifier) were compared, tuned with a Genetic Algorithm, and combined through three ensemble strategies: Weighted Voting via Dirichlet-distributed random search, Stacking, and Greedy Ensemble Selection. Weighted Voting achieved the best validation F1-score (0.6841), with RUSBoost receiving the largest weight (0.6700). A 7-feature subset, selected via SHAP, native feature importance, and linear coefficients, was statistically indistinguishable from the full 22-feature model (McNemar test,  = 0.8388) and was adopted as the final model for parsimony. On the held-out test set, the final model achieved F1 = 0.6776, ROC-AUC = 0.9076, and PR-AUC = 0.6961, with only a 0.0065 gap from validation performance, confirming strong generalization. These results demonstrate that rigorous chronological splitting and formal statistical testing can materially change both the interpretation and the trustworthiness of ensemble classification results in weather-dependent decision support.