Cannot acknowledge message with deserialized message ID

[adam987]

pulsar-client version: 1.12.0
Pulsar version: 4.0.0

We are creating a Pulsar client service that should enable message acknowledgment with a previously serialized message ID.

During our tests:

1. No error is thrown during ID deserialization (calling toString on the deserialized message ID also returns a valid value);
2. No error is thrown during message acknowledgment using acknowledgeId method;
3. The message that should be acknowledged after the process isn't marked to delete by Pulsar and is received again after resubscription when we use the deserialized message ID.

This makes the deserialized message ID unusable for acknowledging the message. When we use message ID retrieved directly from a received message (without serialization/deserialization) acknowledgment by ID works correctly.

Below is a short version of the code that doesn't seem to work even though no error is thrown
consumer.acknowledgeId(MessageId.deserialize(message.getMessageId().serialize()))

Some data (maybe a topic) is probably lost during serialization/deserialization, making the deserialized message ID useless. However, the fact that the acknowledgeId method returns no error seems strange.

Expected behavior:
The deserialized message ID should enable acknowledging a message.

I'm also attaching a link to the repo with the error reproduction
https://github.com/adam987/pulsar-message-id-deserialization

[shibd]

Hi, it's not caused by deserialized msg-id.

You should enable batchIndexAck on both the client and broker side.

[adam987]

Hi, thank you very much for the help. Enabling batchIndexAck on both sides indeed solved the problem. Could you explain why it is needed when deserialized msg-id is used and is not required when msg-id is retrieved directly from a message? To clarify, the same happens regardless of whether we use batchReceive or receive method.

[shibd]

hi,

If you did not enable batchIndexAck

For batch messaging, the pulsar client will record single message ack stats, once all messages has been acked then send ack request to broker.

This status record in the message ID. The deserialized message ID will lose this information, so when it's sent to the broker, the request will be ignored because the broker has not enabled batchIndexAck.