You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I used str.extract_regex() to extract several rows, and using arrow's flatten, and vaex's from_arrays, I composed the result into a DataFrameLocal object.
However, I want to replace the '' values with None, how can I achieve this in a fast way?
Now I can only use .apply(lambda x:None if x=='' else x), which is slow.
update: I found a way: using numpy.place(...). It's faster, but need more memory converting to np.array()
update2: the best way without copying any data, may be using Expression.where():
(df.fieldA=='').where(None, df.fieldA)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I used str.extract_regex() to extract several rows, and using arrow's flatten, and vaex's from_arrays, I composed the result into a DataFrameLocal object.
However, I want to replace the '' values with None, how can I achieve this in a fast way?
Now I can only use .apply(lambda x:None if x=='' else x), which is slow.
update: I found a way: using numpy.place(...). It's faster, but need more memory converting to np.array()
update2: the best way without copying any data, may be using Expression.where():
(df.fieldA=='').where(None, df.fieldA)
Beta Was this translation helpful? Give feedback.
All reactions