Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

named_type: forbid arithmetic operations between unrelated named_types #15170

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

andijcr
Copy link
Contributor

@andijcr andijcr commented Nov 28, 2023

explicitly forbids operations like

named_type<int, struct tag_a>(1) + named_type<int, struct tag_b>(1)

The expression worked because there is an implicit conversion possible from named_type<int, tag> to int, and an operator+(int).

this pr deletes the generic base_named_type from the arithmetic operators overload set and implicitly forbids the above case
to compile.

operations like

named_type<int>(1) + 1

are still allowed. see named_type_test.cc

template<typename Lhs, typename Rhs>
concept arithmetic_ready = requires(Lhs l, Rhs r) {
    l + r;
    l - r;
    l += r;
};

static_assert(arithmetic_ready<named_int, named_int>);
static_assert(arithmetic_ready<named_int, int>);
static_assert(
  !arithmetic_ready<named_int, named_type<int, struct another_tag_t>>);

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.2.x
  • v23.1.x
  • v22.3.x

Release Notes

  • none

@andijcr andijcr marked this pull request as ready for review November 28, 2023 13:16
@andijcr andijcr changed the title Feat/named type integer ops named_type: forbid arithmetic operations between unrelated named_types Nov 28, 2023
@vbotbuildovich
Copy link
Collaborator

new failures in https://buildkite.com/redpanda/redpanda/builds/43646#018cf459-993d-49dc-8615-75483eddfdfe:

"rptest.tests.data_transforms_test.DataTransformsLeadershipChangingTest.test_leadership_changing_randomly"

@andijcr
Copy link
Contributor Author

andijcr commented Jan 11, 2024

tentative new issue:

CI Failure (UndefinedBehaviorSanitizer signed integer overflow with last_term_base_offset) in DataTransformsLeadershipChangingTest.test_leadership_changing_randomly

https://buildkite.com/redpanda/redpanda/builds/43646#018cf459-993d-49dc-8615-75483eddfdfe

Module: rptest.tests.data_transforms_test
Class:  DataTransformsLeadershipChangingTest
Method: test_leadership_changing_randomly
<BadLogLines nodes=docker-rp-7(1) example="SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-047cfb4132eb5c631-1/redpanda/redpanda/src/v/utils/named_type.h:79:39 in">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 142, in wrapped
    redpanda.raise_on_bad_logs(
  File "/root/tests/rptest/services/redpanda.py", line 1327, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-7(1) example="SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-047cfb4132eb5c631-1/redpanda/redpanda/src/v/utils/named_type.h:79:39 in">

this is not from dev, it's from #15170
and named_type is the subject of that pr, so it might be caused by the changes there

but looking at the issue it seems like it might actually be unrelated to the code and potentially in dev

this is the log around the failure

TRACE 2024-01-10 17:44:40,656 [shard 1:main] raft - [group_id:1, {kafka/topic-mxxuaqyofz/0}] consensus.cc:344 - Append entries response: {node_id: {id: {1}, revision: {24}}, target_node_id{id: {2}, revision: {24}}, group: {1}, term:{1}, last_dirty_log_index:{108}, last_flushed_log_index:{108}, last_term_base_offset:{-9223372036854775808}, result: success, may_recover:0}
TRACE 2024-01-10 17:44:40,657 [shard 0:main] raft - [group_id:21, {kafka/topic-vlydmfjgxv/0}] state_machine_manager.cc:129 - [default][rm_stm] applying batch with base 1 and last 82 offsets
TRACE 2024-01-10 17:44:40,657 [shard 1:main] raft - [group_id:1, {kafka/topic-mxxuaqyofz/0}] consensus.cc:442 - Updated node {id: {1}, revision: {24}} last committed log index: 108
TRACE 2024-01-10 17:44:40,657 [shard 1:main] raft - [group_id:1, {kafka/topic-mxxuaqyofz/0}] consensus.cc:568 - Updated node {id: {1}, revision: {24}} match 108 and next 109 indices
TRACE 2024-01-10 17:44:40,659 [shard 0:main] raft - [group_id:15, {kafka/topic-mxxuaqyofz/14}] consensus.cc:344 - Append entries response: {node_id: {id: {3}, revision: {24}}, target_node_id{id: {2}, revision: {24}}, group: {15}, term:{1}, last_dirty_log_index:{115}, last_flushed_log_index:{115}, last_term_base_offset:{-9223372036854775808}, result: success, may_recover:0}
TRACE 2024-01-10 17:44:40,659 [shard 0:main] raft - [group_id:15, {kafka/topic-mxxuaqyofz/14}] consensus.cc:442 - Updated node {id: {3}, revision: {24}} last committed log index: 115
TRACE 2024-01-10 17:44:40,659 [shard 0:main] raft - [group_id:15, {kafka/topic-mxxuaqyofz/14}] consensus.cc:568 - Updated node {id: {3}, revision: {24}} match 115 and next 116 indices
TRACE 2024-01-10 17:44:40,659 [shard 1:main] raft - [group_id:1, {kafka/topic-mxxuaqyofz/0}] consensus.cc:2644 - flushed offset updated: 117
/var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-047cfb4132eb5c631-1/redpanda/redpanda/src/v/utils/named_type.h:79:39: runtime error: signed integer overflow: 110 - -9223372036854775808 cannot be represented in type 'type' (aka 'long')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-047cfb4132eb5c631-1/redpanda/redpanda/src/v/utils/named_type.h:79:39 in 

110 - -9223372036854775808
is derived from
last_term_base_offset:{-9223372036854775808}

and the place of the exception should be

encode_one_delta_array<model::offset>(out, encodee.last_term_base_offset);

calling into
encode_varint_delta(o, v[i - 1], v[i]);

@andijcr
Copy link
Contributor Author

andijcr commented Jan 11, 2024

/ci-repeat dev 1 skip-unit skip-redpanda-build dt-repeat=50 tests/rptest/tests/data_transforms_test.py::DataTransformsLeadershipChangingTest.test_leadership_changing_randomly

@vbotbuildovich
Copy link
Collaborator

Invalid CI repeat count. Must be between 1 and 20.

Workflow run logs.

@andijcr
Copy link
Contributor Author

andijcr commented Jan 11, 2024

/ci-repeat 1 dev skip-unit skip-redpanda-build dt-repeat=50 tests/rptest/tests/data_transforms_test.py::DataTransformsLeadershipChangingTest.test_leadership_changing_randomly

before, the implicit type conversion would make expression like
named_type<int, struct tag_a>(1) + named_type<int, struct tag_b>(1) compile.

now these operations are deleted, while retaining operations like
named_type<int>(1) + 1
@andijcr andijcr force-pushed the feat/named_type_integer_ops branch from 00f4096 to a48a12d Compare March 1, 2024 15:36
@mmaslankaprv mmaslankaprv removed their request for review May 22, 2024 13:27
@dotnwat
Copy link
Member

dotnwat commented Jun 30, 2024

where did we get with this @andijcr ?

@andijcr
Copy link
Contributor Author

andijcr commented Jul 1, 2024

where did we get with this @andijcr ?

want me to resume it for august, for the next release cycle? @dotnwat

@dotnwat
Copy link
Member

dotnwat commented Jul 3, 2024

where did we get with this @andijcr ?

want me to resume it for august, for the next release cycle? @dotnwat

i'm not sure. i mean, i guess it's kind of important, but on the other hand i'm not sure. i tagged you today about it in another PR. it does seems like if we are going ot have named_type then we shoudl at least have it enforce its rules :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants