Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add datapoints.delete method for improved dps delete support #1084

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

erlendvollset
Copy link
Collaborator

@erlendvollset erlendvollset commented Nov 15, 2022

Wanna make this just datapoints.delete() instead. And that can accept timestamps and ranges, as well as Datapoint and Datapoints objects. Then we'll deprecate the other ones in V5 and remove in V6.

@erlendvollset erlendvollset requested review from a team as code owners November 15, 2022 12:02
@erlendvollset erlendvollset changed the title delete dps fix Support same value for end and start in datapoints.delete_range Nov 15, 2022
@erlendvollset erlendvollset added auto-update Will automatically keep PR up to date auto-merge and removed auto-update Will automatically keep PR up to date auto-merge labels Nov 15, 2022
@erlendvollset erlendvollset changed the title Support same value for end and start in datapoints.delete_range Add datapoints.delete_point method for deleting single datapoint Nov 15, 2022
@codecov
Copy link

codecov bot commented Nov 15, 2022

Codecov Report

Merging #1084 (2fd0ad0) into master (2701d10) will increase coverage by 0.01%.
The diff coverage is 100.00%.

❗ Current head 2fd0ad0 differs from pull request most recent head c99f595. Consider uploading reports for the commit c99f595 to get more accurate results

@@            Coverage Diff             @@
##           master    #1084      +/-   ##
==========================================
+ Coverage   91.59%   91.60%   +0.01%     
==========================================
  Files          75       75              
  Lines        7837     7841       +4     
==========================================
+ Hits         7178     7183       +5     
+ Misses        659      658       -1     
Impacted Files Coverage Δ
cognite/client/_api/datapoints.py 98.06% <100.00%> (+0.01%) ⬆️
cognite/client/_version.py 100.00% <100.00%> (ø)
cognite/client/_http_client.py 97.19% <0.00%> (+0.93%) ⬆️

Copy link
Contributor

@amorken amorken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 in principle but maybe adjust docs before merging?

cognite/client/_api/datapoints.py Outdated Show resolved Hide resolved
cognite/client/_api/datapoints.py Outdated Show resolved Hide resolved
cognite/client/_api/datapoints.py Outdated Show resolved Hide resolved
Copy link
Contributor

@haakonvt haakonvt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few thoughts from me 😊

Comment on lines +360 to +365
for point in point_list:
for key in point:
if key not in ("id", "externalId", "timestamp"):
raise AssertionError(
f"Invalid key '{key}' in point. Must contain 'timestamp', and 'id' or 'externalId"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I've seen this type of logic in multiple places in the SDK, could be split out to utils.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do 👍

Comment on lines +363 to +364
raise AssertionError(
f"Invalid key '{key}' in point. Must contain 'timestamp', and 'id' or 'externalId"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KeyError or ValueError?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Some fire-from-the-hip copy-pasting here 😁

...

def delete_point(
self, point: Union[Dict[str, Union[int, str, datetime]], Sequence[Dict[str, Union[int, str, datetime]]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To me it doesn't make much sense to allow str here; as these are all the "unprecise" or relative time offsets like "an hour ago" or "now".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. But this dict also contains the externslId, which has a string value.

...

def delete_point(
self, point: Union[Dict[str, Union[int, str, datetime]], Sequence[Dict[str, Union[int, str, datetime]]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...what I would like to see though was Datapoint and Datapoints supported!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! That's probably much more useful.

def delete_point(
self, point: Union[Dict[str, Union[int, str, datetime]], Sequence[Dict[str, Union[int, str, datetime]]]]
) -> None:
"""Delete a single datapoint from one or more time series.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a suggestion as the current sound like "delete t=200 from ts=a,b and c":

Suggested change
"""Delete a single datapoint from one or more time series.
"""Delete individual datapoints from one or more time series.

id = cast(Optional[int], point.get("id"))
external_id = cast(Optional[str], point.get("externalId"))
valid_range = Identifier.of_either(id, external_id).as_dict()
start = utils._time.timestamp_to_ms(point["timestamp"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May raise KeyError. Add a must-be-present check in loop above?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check above already ensures the existence of this key. Or did you mean smth else?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no it doesn't, it just checks that no other keys are present. I'll make a utility for doing both of these types of checks for dicts.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch!

@@ -346,17 +395,17 @@ def delete_range(
"""
start = utils._time.timestamp_to_ms(start)
end = utils._time.timestamp_to_ms(end)
assert end > start, "end must be larger than start"
assert end >= start, "end must be larger than start"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: revert this. Having them equal doesn't make sense.

@erlendvollset erlendvollset changed the title Add datapoints.delete_point method for deleting single datapoint WIP: Add datapoints.delete_point method for deleting single datapoint Nov 18, 2022
@erlendvollset erlendvollset changed the title WIP: Add datapoints.delete_point method for deleting single datapoint WIP: Add datapoints.delete method for improved dps delete support Nov 18, 2022
@erlendvollset erlendvollset marked this pull request as draft February 16, 2023 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants