High-Performance MQTT Server and MQTT cluster

This project was developed based on surgermq.(https://github.com/surgemq/surgemq)

surgemq is not support cluster and consumer group.

MQTT

According to the MQTT spec:

MQTT is a Client Server publish/subscribe messaging transport protocol. It is light weight, open, simple, and designed so as to be easy to implement. These characteristics make it ideal for use in many situations, including constrained environments such as for communication in Machine to Machine (M2M) and Internet of Things (IoT) contexts where a small code footprint is required and/or network bandwidth is at a premium.

The protocol runs over TCP/IP, or over other network protocols that provide ordered, lossless, bi-directional connections. Its features include:

• Use of the publish/subscribe message pattern which provides one-to-many message distribution and decoupling of applications.
• A messaging transport that is agnostic to the content of the payload.
• Three qualities of service for message delivery:
  • "At most once", where messages are delivered according to the best efforts of the operating environment. Message loss can occur. This level could be used, for example, with ambient sensor data where it does not matter if an individual reading is lost as the next one will be published soon after.
  • "At least once", where messages are assured to arrive but duplicates can occur.
  • "Exactly once", where message are assured to arrive exactly once. This level could be used, for example, with billing systems where duplicate or lost messages could lead to incorrect charges being applied.
• A small transport overhead and protocol exchanges minimized to reduce network traffic.
• A mechanism to notify interested parties when an abnormal disconnection occurs.

There's some very large implementation of MQTT such as Facebook Messenger. There's also an active Eclipse project, Paho, that provides scalable open-source client implementations for many different languages, including C/C++, Java, Python, JavaScript, C# .Net and Go.

Features, Limitations, and Future

Details you can view surgemq.

Performance

It will be done later.

Cluster

Because I don't have experience to develop message queue cluster, so I refer to NATS to do the mqtt cluster.

Architecture diagram

Group

Consumer group.MQTT clients subscribe a regular topic,just one client will receive message in random.

Architecture diagram

Focus

• Clusters use qos 0 when they publish and subscribe topic.I think too much shake hands will reduce the performance of server.
• Clusters's connections is not supported TLS and SSL now.
• Group topic must start with '$g'.Example: $g/:name/xxxxx/xxxx.
• '*' and '+' are allowed in group name. Group name's length < 10. Name is '**123++' is ok.
• Consumer group is not been supported in mqtt cluster.

Examples

Start

  1. Run by binary file
    
      go get github.com/578157900/supermq
      
      cd $GOPATH/src/github.com/578157900/supermq

      # go version > go1.11
      export GOPROXY=https://goproxy.cn
      export GO111MODULE=on

      go build 

      supermq -c appserver.conf
  
  2. Run by docker <standalone> 
    
      ./docker.sh

  3. Start supermq cluster fastly by docker compose <cluster>
    
      docker-compose -f docker-compose-cluster.yaml up -d 

Result

subscribe in 1883 server

publish in 1884 server

Conclusion

• I made this project in short time so i believe this project will have too many bugs.And I hope everyone can help me improve the stability and usability of this service, Thank you!!! Please call me!!!

Contribute

panyingyun