Categorical DataType

[dharik-arsath]

In pandas we usually do data.col.astype('category') but in vaex it does not work, so how to change the string column into categorical column?

and also It will be nice if you implement astype in dataframe than Series. like data.astype({col1:datatype, col2:datatype ..... and so on})

[dharik-arsath]

I think data.viz.categorize(col_name) should work right?

[JovanVeljanoski]

What are you trying to achieve by turning a string column to a category? This is supported in vaex but it might have a different meaning compared to pandas.

Since vaex works fully out of core, if your data is on disk (hdf5, arrow), you don't have to worry about memory issues, which in large part is the reason for using categories in pandas.

In vaex, turning strings into categories can speed up certain operations (like groupby, binby etc.). In any case, it is done like this:

df = df.ordinal_encode('my_column')

This turns (encodes) that column as ints, so operations are faster. You can see the mapping dictionary in df._categories.

Also for now, printing out the df will give you the encoded values. In the future (next major release), this behavior will change and the user will printing / previews will give the original strings. To get the behavior now you can do

df = df._future()

We took this approach to avoid breaking changes (for now).

I hope this helps.

[dharik-arsath]

Hi In Ordinal Encode documentation it says existing column is renamed to hidden column and replaced by numerical column. so how can I get that hidden column, as you said df._future() did not work for me it says attribure error.

[JovanVeljanoski]

Which vaex version do you have? It will only work in the latest one i think..

[dharik-arsath]

I assume you are asking vaexcore and it is 4.2.0

[JovanVeljanoski]

Install the latest version please, and try with that one.
(although I am not 100% sure if that _future was released there, but I think it should be.

[dharik-arsath]

Yeah in vaex-core 4.3.0 post1 it works using df._future. Thanks