Replacing the Standard GP Model with DKL in BoTorch and Evaluate GP Model Fit

[abdsharaf]

Can Deep Kernel Learning (DKL) be used in BoTorch as an alternative to the standard Gaussian Process (GP) model?

I’m considering this because the classical GP model takes quite a long time to fit on my dataset.

Additionally, which performance metrics or model diagnostics are available in BoTorch to assess how well a GP model captures the underlying mapping of the dataset (e.g., predictive accuracy or generalization capability)?

[eytan]

Hi @abdsharaf, How many observations do you have and what is the input dimensionality?

DKL is great for learning a feature extractor in advance, but would only help with large datasets if you use stochastic variational inference. Of it’s not too high dimensional ou could just use PPGPR (I know @nataliemaus makes extensive use of it, but afaik we don’t have any tutorials), perhaps in combination with dkl. There is also now a variational last layer code example which should also scale to larger N, higher D problems. https://botorch.org/docs/v0.15.1/notebooks_community/vbll_thompson_sampling/