-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Add datapoints.delete method for improved dps delete support #1084
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -2,7 +2,7 @@ | |||||
import math | ||||||
import re as regexp | ||||||
from datetime import datetime | ||||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple, Union, cast | ||||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, Set, Tuple, Union, cast, overload | ||||||
|
||||||
import cognite.client.utils._time | ||||||
from cognite.client import utils | ||||||
|
@@ -322,6 +322,55 @@ def insert_multiple(self, datapoints: List[Dict[str, Union[str, int, List]]]) -> | |||||
dps_poster = DatapointsPoster(self) | ||||||
dps_poster.insert(datapoints) | ||||||
|
||||||
@overload | ||||||
def delete_point(self, point: Dict[str, Union[int, str, datetime]]) -> None: | ||||||
... | ||||||
|
||||||
@overload | ||||||
def delete_point(self, point: Sequence[Dict[str, Union[int, str, datetime]]]) -> None: | ||||||
... | ||||||
|
||||||
def delete_point( | ||||||
self, point: Union[Dict[str, Union[int, str, datetime]], Sequence[Dict[str, Union[int, str, datetime]]]] | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ...what I would like to see though was There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good idea! That's probably much more useful. |
||||||
) -> None: | ||||||
"""Delete a single datapoint from one or more time series. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just a suggestion as the current sound like "delete t=200 from ts=a,b and c":
Suggested change
|
||||||
|
||||||
Args: | ||||||
point (Union[Dict[str, Union[int, str, datetime]], Sequence[Dict[str, Union[int, str, datetime]]]]): The point or points to delete | ||||||
|
||||||
Returns: | ||||||
None | ||||||
|
||||||
Examples: | ||||||
|
||||||
Deleting a single datapoint from a single time series:: | ||||||
|
||||||
>>> from cognite.client import CogniteClient | ||||||
>>> c = CogniteClient() | ||||||
>>> c.datapoints.delete_point(point={"id": 1, "timestamp": 12345}) | ||||||
|
||||||
Deleting a single datapoint from multiple time series:: | ||||||
|
||||||
>>> from cognite.client import CogniteClient | ||||||
>>> c = CogniteClient() | ||||||
>>> c.datapoints.delete_point(point=[{"id": 1, "timestamp": 123}, {"externalId": "abc", "timestamp": datetime(year=1992)}]) | ||||||
""" | ||||||
point_list = point if isinstance(point, Sequence) else [point] | ||||||
valid_ranges = [] | ||||||
for point in point_list: | ||||||
for key in point: | ||||||
if key not in ("id", "externalId", "timestamp"): | ||||||
raise AssertionError( | ||||||
f"Invalid key '{key}' in point. Must contain 'timestamp', and 'id' or 'externalId" | ||||||
Comment on lines
+363
to
+364
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep. Some fire-from-the-hip copy-pasting here 😁 |
||||||
) | ||||||
Comment on lines
+360
to
+365
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I've seen this type of logic in multiple places in the SDK, could be split out to utils. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will do 👍 |
||||||
id = cast(Optional[int], point.get("id")) | ||||||
external_id = cast(Optional[str], point.get("externalId")) | ||||||
valid_range = Identifier.of_either(id, external_id).as_dict() | ||||||
start = utils._time.timestamp_to_ms(point["timestamp"]) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. May raise There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The check above already ensures the existence of this key. Or did you mean smth else? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah no it doesn't, it just checks that no other keys are present. I'll make a utility for doing both of these types of checks for dicts. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great catch! |
||||||
valid_range.update({"start": start, "end": start + 1}) | ||||||
valid_ranges.append(valid_range) | ||||||
self.delete_ranges(ranges=valid_ranges) | ||||||
|
||||||
def delete_range( | ||||||
self, start: Union[int, str, datetime], end: Union[int, str, datetime], id: int = None, external_id: str = None | ||||||
) -> None: | ||||||
|
@@ -346,17 +395,17 @@ def delete_range( | |||||
""" | ||||||
start = utils._time.timestamp_to_ms(start) | ||||||
end = utils._time.timestamp_to_ms(end) | ||||||
assert end > start, "end must be larger than start" | ||||||
assert end >= start, "end must be larger than start" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note to self: revert this. Having them equal doesn't make sense. |
||||||
|
||||||
delete_dps_object = Identifier.of_either(id, external_id).as_dict() | ||||||
delete_dps_object.update({"inclusiveBegin": start, "exclusiveEnd": end}) | ||||||
self._delete_datapoints_ranges([delete_dps_object]) | ||||||
|
||||||
def delete_ranges(self, ranges: List[Dict[str, Any]]) -> None: | ||||||
def delete_ranges(self, ranges: List[Dict[str, Union[int, str, datetime]]]) -> None: | ||||||
"""`Delete a range of datapoints from multiple time series. <https://docs.cognite.com/api/v1/#operation/deleteDatapoints>`_ | ||||||
|
||||||
Args: | ||||||
ranges (List[Dict[str, Any]]): The list of datapoint ids along with time range to delete. See examples below. | ||||||
ranges (List[Dict[str, Union[int, str, datetime]): The list of datapoint ids along with time range to delete. See examples below. | ||||||
|
||||||
Returns: | ||||||
None | ||||||
|
@@ -378,8 +427,8 @@ def delete_ranges(self, ranges: List[Dict[str, Any]]) -> None: | |||||
raise AssertionError( | ||||||
"Invalid key '{}' in range. Must contain 'start', 'end', and 'id' or 'externalId".format(key) | ||||||
) | ||||||
id = range.get("id") | ||||||
external_id = range.get("externalId") | ||||||
id = cast(Optional[int], range.get("id")) | ||||||
external_id = cast(Optional[str], range.get("externalId")) | ||||||
valid_range = Identifier.of_either(id, external_id).as_dict() | ||||||
start = utils._time.timestamp_to_ms(range["start"]) | ||||||
end = utils._time.timestamp_to_ms(range["end"]) | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
__version__ = "4.11.1" | ||
__version__ = "4.11.2" | ||
__api_subversion__ = "V20220125" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To me it doesn't make much sense to allow
str
here; as these are all the "unprecise" or relative time offsets like "an hour ago" or "now".There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. But this dict also contains the externslId, which has a string value.