Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset copy in non-inplace changes #68

Open
alessiamarcolini opened this issue Sep 23, 2020 · 1 comment
Open

Dataset copy in non-inplace changes #68

alessiamarcolini opened this issue Sep 23, 2020 · 1 comment
Labels
enhancement New feature or request

Comments

@alessiamarcolini
Copy link
Contributor

My idea would be to use Pandas for the copy of the DataFrame (which is the biggest object in memory for the instance).

Looking at the docs they say (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.copy.html):

When deep=True, data is copied but actual Python objects will not be copied recursively, only the reference to the object. This is in contrast to copy.deepcopy in the Standard Library, which recursively copies object data (see examples below

So it could be more memory-efficient.
According to https://stackoverflow.com/questions/9058305/getting-attributes-of-a-class , a possible suggestion for implementation could be gather the attributes except for _df attribute (that will be copied with pd.copy()) and create a new instance with the copy.deepcopy() of those attributes (not _df):

for attribute, value in self.__dict__.items():
    print(attribute, '=', value)
copied_d = Dataset()
copied_d.__class__.__dict__ = {key: copy.deepcopy(value) for (key, value) in original_d.__class__.__dict__.items() if key != '_df'}
copied_d._df = original_d._df.copy()

What do you think? Does it make sense?

Originally posted by @lorenz-gorini in #67 (comment)

@lorenz-gorini
Copy link
Member

Requires #38 and #39 to be fixed (removing the possibility to create a Dataset from either an existing pd.DataFrame or from a CSV file)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants