Early stopping with gblinear doesn't save the best model for subsequent prediction

[sktin]

Code to replicate:

import xgboost as xgb
print(F'{xgb.__version__=}')

from sklearn.datasets import make_classification
from xgboost import XGBClassifier
from sklearn.metrics import roc_auc_score

X, y = make_classification(50000, random_state=0)
model = XGBClassifier(
    booster='gblinear', updater='coord_descent',
    eval_metric='auc', eta=0.01, 
    early_stopping_rounds=10, n_estimators=1000000, 
    random_state=0, n_jobs=4
)
model.fit(X, y, eval_set=[(X, y)], verbose=1)
print(F'{model.best_iteration=}, {model.best_score=}')
print(roc_auc_score(y, model.predict_proba(X)[:,1]))

Output:

xgb.__version__='2.1.1'
[0]	validation_0-auc:0.93851
[1]	validation_0-auc:0.93851
[2]	validation_0-auc:0.93851
[3]	validation_0-auc:0.93851
[4]	validation_0-auc:0.93851
[5]	validation_0-auc:0.93851
[6]	validation_0-auc:0.93851
[7]	validation_0-auc:0.93851
[8]	validation_0-auc:0.93851
[9]	validation_0-auc:0.93851
[10]	validation_0-auc:0.93851
[11]	validation_0-auc:0.93851
[12]	validation_0-auc:0.93851
[13]	validation_0-auc:0.93851
[14]	validation_0-auc:0.93851
[15]	validation_0-auc:0.93850
[16]	validation_0-auc:0.93850
model.best_iteration=6, model.best_score=0.9385135544606076
0.9385032096436587

iteration_range has no effect.

print(roc_auc_score(y, model.predict_proba(X, iteration_range=(0,7))[:,1]))
print(roc_auc_score(y, model.predict_proba(X, iteration_range=(0,1000000))[:,1]))
print(roc_auc_score(y, model.predict_proba(X, iteration_range=(0,1))[:,1]))

Output:

0.9385032096436587
0.9385032096436587
0.9385032096436587

The only workaround I can think of is re-fitting with the best_iteration found.

model = XGBClassifier(
    booster='gblinear', updater='coord_descent',
    eval_metric='auc', eta=0.01, 
    n_estimators=model.best_iteration+1, 
    random_state=0, n_jobs=4
)
model.fit(X, y, eval_set=[(X, y)], verbose=1)
print(roc_auc_score(y, model.predict_proba(X)[:,1]))

Output:

[0]	validation_0-auc:0.93851
[1]	validation_0-auc:0.93851
[2]	validation_0-auc:0.93851
[3]	validation_0-auc:0.93851
[4]	validation_0-auc:0.93851
[5]	validation_0-auc:0.93851
[6]	validation_0-auc:0.93851
0.9385135544606077