Several protocol violations or bugs in HiveMQ

[panhao410]

Describe the bug
Hi, I found something on the HiveMQ that is contrary to the protocol specification description (protocol violation or logic bug).
For tracking purposes, I will report all results under this Issue.

Environment Details

• HiveMQ version 4.24.0 (HiveMQ Enterprise Server)
• Operating system and version: Ubuntu 20.04

Client SDK
If possible include the mqtt sdk you used to connect to hivemq
Minimal C test cases are perferred.

Additional context
Add any other context about the problem here.

[panhao410]

According to the specification of MQTTv3.1.1:

If the User Name Flag is set to 1, a User Name MUST be present in the payload .

Replay such packet:

echo 102700044d5154540482d05000193349503972474f63796a50496e32704e446b3551484a5252750000 | xxd -p -r | nc 127.0.0.1 1883

[panhao410]

According to the specification of MQTTv3.1.1:

If the Will Flag is set to 1, the Will QoS and Will Retain fields in the Connect Flags will be used by the Server, and the Will Topic and Will Message fields MUST be present in the payload.

Replay such packet:

echo 102700044d5154540406b19c000669523069336100136f324373774a324b527065374e614a38726554 | xxd -p -r | nc 127.0.0.1 1883

[panhao410]

According to the specification of MQTTv3.1.1:

SUBSCRIBE, UNSUBSCRIBE, and PUBLISH (in cases where QoS > 0) Control Packets MUST contain a non-zero 16-bit Packet Identifier.

Replay such packet:

echo 105d00044d515454040c8e3c0018546a37786f3258556551443047644e385950706d51613251001c54374578706f4e686d4679485874384b4c483872554247764935385500196f65486351376f6a694d48596355524d3341533052686b7164a2110000000d357a386333384e6f416a54305a
 | xxd -p -r | nc 127.0.0.1 1883

[SgtSilvio]

Hi @panhao410
you mentioned "NanoMQ", is this a typo?
HiveMQ does not offer a product called "NanoMQ".

[SgtSilvio]

Regarding your first comment:

If the User Name Flag is set to 1, a User Name MUST be present in the payload .

This is correct behavior, as the user name is actually set; it only has a length of 0, which is not disallowed.

[panhao410]

Hi @panhao410 you mentioned "NanoMQ", is this a typo? HiveMQ does not offer a product called "NanoMQ".

Some spelling mistakes, sorry. I have corrected them.

[songxpu]

Hi, I also found some new bugs here.

# MQTTv3.1.1
The User Name MUST be a UTF-8 encoded string as defined in Section 1.5.3 [MQTT-3.1.3-11].

Replay such packet:

102900044d515454048049f4000b45696a5033694430694155001062313166564374ff74614b4e36303536

It is worth mentioning that HiveMQ correctly terminates the connection when dealing with CONNECT packets such as MQTTv5.0 and prompts: 'Sent CONNECT with malformed username. ' , but it seems to be missing on MQTT v3.1.1.

[songxpu]

# MQTT v5
3.3.2.3.2 Payload Format Indicator

· 0 (0x00) Byte Indicates that the Payload is unspecified bytes, which is equivalent to not sending a Payload Format Indicator.
· 1 (0x01) Byte Indicates that the Payload is UTF-8 Encoded Character Data. The UTF-8 data in the Payload MUST be well-formed UTF-8 as defined by the Unicode specification [Unicode] and restated in RFC 3629 [RFC3629].

According to this description, when the Payload Format Indicator in the PUBLISH message is set to 1, the payload in the PUBLISH message must be UTF-8 encoded. However, it appears that HiveMQ does not validate this.

PoC:

echo 106e00044d515454056e866e0711c4f23d2e17010012764d78635176754e4b6163615164477247521d0300027a4e260009577852505166527043000a6e467067664149535563000f487a4134464c4d3352526c6231486c00104e364958486246414654713051374b79000559304c56433d3900087751586170725838415f060101090001b8423663546c627950717a306b363233664d3249426c36717a6a5a474546456a6f3845363038fe | xxd -r -p | nc 127.0.0.1 1883

[songxpu]

Similarity,

# MQTT v5
3.1.3.2.3 Payload Format Indicator

Followed by the value of the Payload Format Indicator, either of:
· 0 (0x00) Byte Indicates that the Will Message is unspecified bytes, which is equivalent to not sending a Payload Format Indicator.
· 1 (0x01) Byte Indicates that the Will Message is UTF-8 Encoded Character Data. The UTF-8 data in the Payload MUST be well-formed UTF-8 as defined by the Unicode specification [Unicode] and restated in RFC 3629 [RFC3629]

According to this description, when the Payload Format Indicator in the CONNECT message is set to 1, the will message in the CONNECT message must be UTF-8 encoded. However, it appears that HiveMQ does not validate this.

POC:

106300044d51545405a6fd850c11987da8451900270b5434c60004534d763907010118e7f5bf8600104371794d4a3374706e773872744b364300126c397a7a71763353325647714f6a475951fe0016354278463574494f433735413474654d745869475467

[Remit]

Hi @songxpu and @panhao410,

Thank you for reporting the issues with HiveMQ CE! We’ve looked over them. Below is the summary.

• MQTT3.1.1 CONNECT packet that has will flag but no will payload present accepted by the broker -> an issue that we will fix
• MQTT3.1.1 CONNECT packet that has non-UTF8 username accepted by the broker -> an issue that we will fix
• MQTT3.1.1 UNSUBSCRIBE packet with 0 packet ID accepted by the broker -> an issue that we will fix

@songxpu UTF8 validation of the payload for will PUBLISH and ordinary PUBLISH on the server is allowed by MQTT protocol but NOT prescribed.

For CONNECT packet, the protocol spec states: “The Server MAY validate that the Will Message is of the format indicated, and if it is not send a CONNACK with the Reason Code of 0x99 (Payload format invalid) as described in section 4.13.”.
For PUBLISH packet, the protocol spec states: “The receiver MAY validate that the Payload is of the format indicated, and if it is not send a PUBACK, PUBREC, or DISCONNECT with Reason Code of 0x99 (Payload format invalid) as described in section 4.13.  Refer to section 5.4.9 for information about security issues in validating the payload format.”

Basically, the paragraphs 3.1.3.2.3 and 3.3.2.3.2 state the following: the MQTT client MUST conform to the UTF8 format whereas the broker MAY validate the will and PUBLISH payloads that client had sent. In your case, the client misbehaves by not conforming to the MQTT protocol, but the broker is allowed to accept these payloads nevertheless.

If you want to enable validation of PUBLISH payloads, then you can configure it via the config.xml file. You should be able to do this by adding the following config into the XML:

<security>
   <payload-format-validation>
      <enabled>true</enabled>
   </payload-format-validation>
</security>

You can see the defaults in SecurityConfigurationService

I hope, this helps.