Custom objective and evaluation function

[rbeeli]

The Sklearn API provides a simple way of specifying custom objective functions and evaluation metrics:

********* Sklearn API **********
# default lightgbm model with sklearn api
gbm = lightgbm.LGBMRegressor() 

# updating objective function to custom
# default is "regression"
# also adding metrics to check different scores
gbm.set_params(**{'objective': custom_asymmetric_train}, metrics = ["mse", 'mae'])

# fitting model 
gbm.fit(
    X_train,
    y_train,
    eval_set=[(X_valid, y_valid)],
    eval_metric=custom_asymmetric_valid,
    verbose=False,
)

Source: https://towardsdatascience.com/custom-loss-functions-for-gradient-boosting-f79c1b40466d

Is the same possible with LightGBM.jl? If not, can this feature be added?

[yalwan-iqvia]

Hi @rbeeli thanks for raising this issue.

This request is not super straightforward or high on our priorities list. With that said we can take a look at what is required to implement these features, but I'm not sure we can turn this one around quickly.

[yalwan-iqvia]

Notes:
Custom objective requires implementing currently unimplemented C-API call LGBM_BoosterUpdateOneIterCustom
Custom metrics: Needs to rejig fit!/train! calls, possibly needs to implement some C-API functions to get predictions without calling predict directly, needs more investigation

[yalwan-iqvia]

To support custom metrics without calling predict, we can use LGBM_BoosterGetPredict -- this is already implemented

[yaxxie]

#114 make partial progress towards this request