Utils to format extraction of index and nunique of categorical columns

[ekamioka]

Implement auxiliary function to extract dict of indexes and unique values/dims

Usage:
    cat_dims = get_idxs_dims(df_train, cat_cols=categorical_features)

    clf = TabNetClassifier(cat_idxs=[x for x, _ in cat_dims.items()],
                                          cat_dims=[x for _, x in cat_dims.items()],
                                          )

maybe towards "cat_idxs and cat_dims are not required anymore"
    clf = TabNetClassifier(cat_cols=categorical_cols) 

What kind of change does this PR introduce?
Auxiliary function to allow internal/encapsulated extraction of indexes receiving only a list of categorical columns

Does this PR introduce a breaking change?
So far, nops

What needs to be documented once your changes are merged?
Please check comments in the code

[Optimox]

Thanks for your PR.

I think it's indeed an interesting feature to add to have some classical preprocessing function implemented inside the repo.

However, I think this should provide a bit more than just cat_dims.
It should probably give cat_idxs (as shown in your code above) and also cat_emb_dims with predefined methods like 'log' which whould create emb_dim = int(log(cat_dim) 'max_50' which would do emb_dim = min(50, cat_dim / 2) or something like that.

[ekamioka]

Thanks for your PR.

I think it's indeed an interesting feature to add to have some classical preprocessing function implemented inside the repo.

However, I think this should provide a bit more than just cat_dims.
It should probably give cat_idxs (as shown in your code above) and also cat_emb_dims with predefined methods like 'log' which whould create emb_dim = int(log(cat_dim) 'max_50' which would do emb_dim = min(50, cat_dim / 2) or something like that.

Agree, I am still experimenting with tabnet but will do some drafts in parallel before committing to this PR. Thanks Seb.

[Optimox]

@ekamioka are you still willing to improve on this? Or should I close it? I opened an issue on this topic #249 so that this idea won't be lost anyway.

[ekamioka]

@ekamioka are you still willing to improve on this? Or should I close it? I opened an issue on this topic #249 so that this idea won't be lost anyway.

yes, pushing soon. thanks for opening the issue.