Feature Importance and Effect on Ensemble_Stacked Model

[tharnato]

First of all, thank you for creating this incredibly impressive AutoML package. This package has greatly assisted me in optimization and the search for the best models. I'm just wondering if there's a way to determine the feature importance of the best model, specifically in the case of the Ensemble_Stacked?

I've attempted to use SHAP, but it appears that SHAP only accommodates one model, requiring us to manually find its parameters from the best model. Is there a method for us to observe the significance of these features and their impact, both positive and negative, on the target variable?

Thank you very much!

[pplonski]

Hi @tharnato,

You should be able to train models with AutoML parameter explain_level=2 and it will produce the features importance for every model. Alternative might be to try train models manually and apply some features importance technique.

From my experience, you don't need to apply features importance for each model, or best model. You can get good approximation of which features are important by computing features importance for some good model (just select well performing one and compute importance).

You can also apply features selection when training AutoML. Please set argument features_selection=True when constructing AutoML and you will get list of features to drop. Dropped features will have importance lower than a random feature. Here is description of our approach https://mljar.com/automated-machine-learning/feature-selection/

[tharnato]

i see, so approxiamation from good model should be enough to represent the feature importance of best model. Then how about if we want to know how a feature going to affect the target variable ? like the the relationship between features and the target variable

[pplonski]

You can try to compute some approximation on how features affect target. You can compute correlation between features and target or you can train some simple model and check how change in the feature (random shuffling) affects prediction performance.

[Sehran-Hassan]

How to propose R code for any machine learning algorithm like, how to select the minimum number of genes using any machine learning model.

[pplonski]

Hi @Sehran-Hassan,

How are you? Could you provide more details about your use case? You can also send me email at [email protected]