# Airline Satisfaction: Model Validation

[Read the case study](index.html) · [View validation results](results.html). An exploratory model-validation case using Kaggle Playground S6E10 generated competition data.

## Retained evidence

`evidence/catboost-review.json` preserves Experiment 75's saved metrics. `evidence/catboost-experiment.py` preserves the confirmation code. Fold 2 AUC: baseline 0.9612149528, blend 0.9612867352. Fold 3: baseline 0.9609630528, blend 0.9609970996. Fixed 30% CatBoost weight; three-fold blend mean AUC 0.9611053056. AUC is not accuracy.

Fold 1 selected the model/weight. Other folds have prior screening history and early stopping uses their labels. There is no untouched final-test estimate. Saved review metrics were checked for arithmetic consistency; training was not rerun for this release. The later submitted blend (Experiment 79) scored 0.96066 public ROC AUC, below the previous own-model submission of 0.96071. The saved account best is 0.96188 from a separate community-assisted ensemble; it is not the CatBoost result. No current rank or final private-test result is claimed.

[Inspect the saved submission review](evidence/submission-review.json).

## Reproduction boundary

The script requires the original Kaggle runtime’s `X_65`, fold definitions, target encodings and baseline out-of-fold predictions. It is an inspection extract, not a standalone pipeline. Raw competition data, models, predictions and unrelated community artifacts are excluded. Fresh evaluation requires rebuilding the feature environment under competition terms.

## Sources and attribution

- [Competition](https://www.kaggle.com/competitions/playground-series-s6e10)
- [Original public dataset](https://www.kaggle.com/datasets/teejmahal20/airline-passenger-satisfaction)
- [Route-feature inspiration](https://www.kaggle.com/code/busyaprime/s6e10-route-or-distance-both-lb-0-96169)
- [LightGBM feature inspiration](https://www.kaggle.com/code/evgendvorkin/s6e10-lightgbm-cv-5-folds-0-96102)
- A separate community-assisted ensemble used [Golem OOF community predictions](https://www.kaggle.com/datasets/dariushafshar/s6e10-golem-oof-library). Its score is not a CatBoost result or independent implementation claim.

## Credits

**Project author.** Lesley Maore. This case documents the retained Airline Satisfaction modelling experiment in my portfolio.

**Data providers.** Kaggle Playground Series S6E10 competition data; the original public airline-passenger-satisfaction dataset is credited in Sources and attribution above.

**Community references.** Route-feature and LightGBM feature inspiration, and the separate Golem OOF ensemble reference, are linked above. Their contributions are distinct from the saved CatBoost comparison shown here.

AI-assisted project.