Pickle support?

[devforfu]

Would it be reasonable to have a custom pickling magic method? Something like this:

class DataFrameSummary:
    ...

    def __getstate__(self):
         return {'length': self.length, 
                 'column_stats': self.column_stats,
                 'corr': self.corr}
   
    def __setstate__(self, state):
        self.length = state['length']
        self.column_stats = state['column_stats']
        self.corr = state['corr']

I guess that usually, you don't need to store df property because it is probably already stored somewhere. Or it could be an optional behavior depending on __init__ parameters.

What do you think?

[mmourafiq]

I am not sure what would be the use case? Faster access or caching?

[devforfu]

In my case, it was for the purpose of restoring the state from the previous run. I am using Jupyter notebooks to carry out some data analysis, and the dataset is huge enough to take a few minutes to compute summary statistics. So I am doing something like this:

temp_summary = temp_dir/'summary.pickle'
if temp_summary.exists():
    print(f'Loading previously saved summary: {temp_summary}')
    summary = pickle.load(temp_summary.open('rb'))
else:
    print('Generating summary statistics')
    summary = DataFrameSummary(data)
    print('Saving summary into pickle file...')
    summary.df = None  # don't need to save the dataset itself
    with temp_summary.open('wb') as file:
        pickle.dump(summary, file)

Not a bit deal but requires additional step to save summary onto disk :) Or even better, it could be something like summary.to_pickle() and summary.read_pickle(). Does it make sense?

[mmourafiq]

A new persisting and viewing logic is under way.