SAVED EXPERIMENT 75 · EXPLORATORY VALIDATION
What Did the Model Add?
A fixed 30% CatBoost blend improved the saved comparison scores slightly. The later competition submission did not improve on the previous own-model submission.
| Fold | Baseline | CatBoost | 70/30 blend | Blend gain |
|---|---|---|---|---|
| Fold 2 | 0.961215 | 0.961069 | 0.961287 | +0.000072 |
| Fold 3 | 0.960963 | 0.960730 | 0.960997 | +0.000034 |
How to read the result
CatBoost scores lower on its own in these two folds, yet its predictions add complementary information to the baseline. The gain is small. Fold 1 selected the model and weight; folds 2 and 3 have prior screening history, and early stopping uses their labels.
The saved three-fold mean blend AUC is 0.961105. This is an exploratory cross-validation result. A new untouched validation design is needed before claiming a reliable generalisation improvement. A later submission of this blend (Experiment 79) scored 0.96066 public ROC AUC, versus 0.96071 for the previous own-model submission. The saved account best of 0.96188 belongs to a separate community-assisted ensemble, not this CatBoost comparison. No current rank or final private-test score is claimed.
What happened next?
Later model screening also failed to justify replacing the retained reference. Experiment 90 had a mean validation change of −0.00000141 ROC AUC and was not advanced or submitted. Its validation data had prior exploration history, so this is a stopping decision, not an independent test result.