fix: allow skipping per message size check

[ae-govau]

This is useful when producer compression is enabled, as it allows for much larger messages to be sent to the broker (providing the entire batch they are in stays small enough).

The previous workaround suggested in #2142 causes other ill-effects related to message batching (such as in #2797).

Fixes #2142, probably also #2797 and may help mitigate not having merged #2851.

It's not clear how easy it is to write a test for this but happy to try if we're happy enough with this approach.

[ae-govau]

This still doesn't solve everything -

sarama/produce_set.go

Lines 253 to 255 in 0ab2bb7

	case ps.msgs[msg.Topic] != nil && ps.msgs[msg.Topic][msg.Partition] != nil &&
	ps.msgs[msg.Topic][msg.Partition].bufferBytes+msg.ByteSize(version) >= ps.parent.conf.Producer.MaxMessageBytes:
	return true

still is checking a before-compression amount for calculating the batch, but I think the above is probably better than erroneously returning errors for small messages when MaxMessageBytes is set too high.

[puellanivis]

The phrasing of “skip” here is potentially confusing, especially when batch processing. Not sure there’s a better phrasing possible though.

PS: Maybe Disable?

[github-actions]

Thank you for your contribution! However, this pull request has not had any activity in the past 90 days and will be closed in 30 days if no updates occur.
If you believe the changes are still valid then please verify your branch has no conflicts with main and rebase if needed. If you are awaiting a (re-)review then please let us know.