4.1 Introduction

Massive IoT interconnects an enormous number of devices and services over the Internet for a broad range of applications. The IoT devices are typically connected to the Internet via Internet Protocol (IP).

The application-layer protocols such as HyperText Transfer Protocol (HTTP) are primarily used for communication on browser-based clients, which typically run on relatively high-capability devices, such as smartphones that use relatively high-bandwidth communications channels (e.g., LTE). On the other hand, IoT protocols are set of rules suitable for IoT applications to ensure that data sent from an IoT device gets read and understood by another device. IoT protocols also need to ensure optimum security of the exchanged information between connected devices. Depending on the IoT devices and their applications, there are different application-layer IoT protocols. The application layer is actually an interface between the user and the IoT device. A specific IoT use case may require a specific application-layer data communication protocol and an IoT-centric communications stack.

There are many IoT protocols available to choose for different applications and requirements. For example, some of these protocols are specifically designed to satisfy the requirements for fast and reliable business transactions or some other are optimized to meet the requirements of data collection such as sensor updates in constrained networks. Some other IoT protocols are suitable for applications where Instant Messaging (IM) and online presence detection is needed [5]. Each of these protocols have their own advantages and disadvantages when dealing with different IoT scenarios. This is why it is important to understand the characteristics of various IoT protocols to be able to choose the best-fil protocol for your use case.

Three widely accepted and key IoT protocols include Message Queue Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), and Advanced Message Queuing Protocol (AMQP). We first introduce each of these three protocols in detail and then present a general comparison between them to gain insight into the strengths and limitations of each protocol.

4.2  Message Queue Telemetry Transport (MQTT) Protocol

MQTT is a lightweight and scalable publish/subscribe IoT messaging protocol that can be easily implemented to relay data through a central server called the broker. This protocol is primarily used for constrained networks that provide connections to remote locations and is ideal for small devices that require efficient bandwidth and battery use such as car sensors, smoke detectors, smart watches, and text-based messaging applications. Compared to HTTP, MQTT is faster, has less overhead and power consumption. The other difference to HTTP is that in MQTT, a client does not have to pull the information it needs, but if there is new data to be sent, the server (broker) pushes the information to the client.

In MQTT, new devices can be easily added without touching the existing infrastructure (multiple publishers and multiple subscribers). Since the new devices only communicate with the broker, they do not need to be compatible with the other devices.

The broker acts like a post office. Instead of a device, sending the messages to or getting them from another device directly (peer-to-peer), the client (publisher) sends the messages to the post office (broker) and then it is forwarded to everyone who needs the message (subscribers). The difference is that MQTT uses subject line called “topic” instead of addresses i.e., everyone who needs a copy of that message subscribes to that topic. A topic is a simple string that can have more hierarchy levels, which are separated by a slash (e.g. sensors/temperature). Note that topics are case sensitive.

In MQTT, the IoT devices are interconnected via the broker, meaning that there is no direct connection between clients and the broker relays the messages between clients. Unlike most communication protocols, this message relay is not restricted to a one-to-one model. The message uses a topic for unique identification and is not targeted into a single device. Hence, it is possible to have many devices subscribe to the same topic, which can realize a one-to-many model. Similarly, a many-to-many model multiple devices can be publishing the same topic and have one or multiple subscribers. The broker handles all of these scenarios by using topics to manage the messages.

MQTT is a bi-directional communication protocol where each client can both produce and consume data by publishing messages and subscribing to topics. The big advantage of this two-way communication is that the IoT devices can send sensor data and at the same time receive configuration information and control commands.

A.  MQTT Connection

MQTT uses three levels of Quality of Service (QoS) to ensure its message reliability. Table4.1 shows these three QoS levels including fire and forget (Level 0), acknowledged delivery (Level 1), and assured delivery (Level 2).

Table4.1. MQTT QoS levels

MQTT requires reliable, lossless, and in-order delivery of packets, and thus, it is a TCP/IP-based protocol where a TCP connection needs to be established prior to transferring the data. MQTT is a three-phase handshake communication that needs to first establish the TCP connection. Second, the MQTT connection is established and data is published. Finally, the TCP connection is terminated.

TCP connection establishment

A three-way handshake is used to establish the TCP connection. First, the client sends SYN request to indicate that it wants to communicate with the server (broker) which is always in the listening mode. Then, this request is acknowledged by the server by sending a [SYN, ACK] packet to let the client know that the server has received the connection request packet and is ready to exchange information with the client. Finally, the client sends ACK to make the server aware that the client has received the acknowledgment of connection request and is ready to transfer data with the server.  

MQTT connection establishment and data publish

There are different MQTT publisher handshake processes for different QoS levels. For QoS level 0, after the TCP connection is successfully established, the publisher sends the connect request packet to the server (broker). Once the connection request packet is received by the server, it sends an acknowledgment packet to the publisher to show that the broker is ready for communication. Then, the publisher starts publishing the message. Finally, the publisher disconnects from the server by sending a disconnect request packet.

TCP connection termination

After the server receives the disconnect request packet, it sends two packets to the client. One ACK packet to acknowledge the reception of disconnect request packet, and the second [FIN, ACK] packet to let the client know that the broker also wants to terminate the connection. The last packet in this phase is an acknowledgement sent by the client.

B.  Security of MQTT protocol

Most IoT devices lack the necessary network resources to manage their connections in a secure manner. The central broker in the MQTT architecture increases the security by decoupling the devices and applications. In this architecture, all devices have to authenticate against a central security location (the broker) and only one port is required to be open to the broker (the secure port 8883). MQTT provides this security using Transport Layer Security (TLS) encryption, and username/password protected connections. Moreover, each client is unaware of IP addresses and domains of other devices. These security mechanisms are configured on the MQTT broker and then the client will need to comply with the security requirements. This is why the capabilities of the MQTT client should be taken into account when planning and implementing security measures on the broker.

There are three ways that a broker can verify the identity of an MQTT client:

• Client IDs: All MQTT clients must provide a Client ID, which is used to link the MQTT topic to the client and to the TCP connection when a client subscribes to a topic.
• Usernames and passwords: Before a connection is established, the MQTT broker can require a valid username and password from the client. Note that this username/password combination is not encrypted or secured and is simply transmitted in plain text.
• Client Certificates: This most secure method to authenticate the clients is typically very difficult to implement because the certificates need to be deployed and managed on many clients. This authentication method could be suitable for small number of clients with strictly high security needs.

C.  Transport Layer Security (TLS)

TLS or as it is more commonly known Secure Socket Layer (SSL) technology is a part of TCP/IP protocol (not MQTT) and provides an encrypted pipe to protect all parts of MQTT messages including the payload. This technology needs to be supported by the client as well and may not be available on cheap and simple IoT devices. Note that data can be encrypted end-to-end at the application layer without configuring the broker. However, passwords cannot be protected because the broker is not involved.

D.  MQTT broker

Hosting an MQTT broker

There are two main options to host an MQTT broker. The MQTT broker can be hosted on a locally installed or a cloud-based server. In the former, MQTT broker is installed on one’s server hardware. Depending on the requirements of each broker, one can choose from many free and open source MQTT brokers. Table4.2 below shows some of these available brokers.

Table4.2. Locally installed brokers

Many companies such as Google, Amazon, Microsoft, and IBM provide cloud-based MQTT server/brokers. For example, Google Cloud IoT Core provides the MQTT protocol through running a managed broker linking to the port “mqtt.googleapis.com:8883”. Note that Port 8883 is the standard TCP port reserved secure MQTT connections.

Reliability of MQTT broker

In MQTT, the broker is the central point of communication between all devices, which raises concern about having a single source of failure. To increase the reliability of the communication, most broker software and clients support automatic handover to a backup/redundant broker if the primary server fails. The software can also be set up to share the load of clients across multiple servers on cite, in the cloud, or a combination of both.

Client status certificates

In MQTT, the network load is reduced by having the client send data only when there is a change in the values. The broker retains the published messages and sends them out to new subscribing clients. The MQTT brokers can be aware of the state of their clients and communicate that state to other clients using birth and death certificates and the Last Will and Testament (LWT) feature. MQTT clients can register an LWT message that will be sent by the broker if they suddenly disconnect. These messages can be used to let the subscribers know when a device disconnects. When a publishing client connects to a broker, it issues a birth certificate along with its LWT payload to the broker. The broker will notify any subscribing client to the same topic as the publishing client whenever the publishing client is offline or online. To ensure that the stale data is not delivered to the subscribing client if the publishing client is offline, the broker provides the subscribing client with the LWT payload.  This is accomplished using a pre-configured ‘keep alive timer’ or ‘heart beat check’ between broker and publishing clients.  As a result, the subscribing clients are always aware whether the publishing client is online, or when the client went offline, and what was the last known value.

4.3 Constrained Application Protocol (CoAP)

The Constrained Application Protocol (CoAP) is an application-layer protocol developed by the Internet Engineering Task Force (IETF) as an extremely lightweight communications protocol stack suitable for resource-constrained devices. CoAP can be implemented over User Datagram Protocol (UDP) and is designed for devices with limited capacity to connect in Light-Weight Machine-to-Machine (LWM2M) communication. LWM2M enables remote management and control of IoT devices using a streamlined managed objects model and provides interfaces for securely monitoring and administering devices. CoAP uses the GET, PUT, POST, and DELETE methods and response codes similar to, but not exactly like, HTTP. Therefore, it is easy to map CoAP traffic from devices to the Representational State Transfer-ful (RESTful) Application Program Interface (API) logic. A RESTful API is an API that utilizes HTTP requests to GET, PUT, POST and DELETE data. An API for a website is a code that allows two software programs to exchange information between them.

CoAP uses the resource model mapped to the Universal Resource Identifier (URI) instead of MQTT topics. However, there is a similarity between the CoAP URIs and MQTT topics. For example, sensor devices publishing their sensor information to a server could be described in the following manner:

CoAP sensor publishing to a CoAP server

URI: coap://devices/sensors/temperature

MQTT client publishing to a sensor queue on the broker

topic: “/devices/sensors/temperature”

In simple terms, the operation of CoAP can be summarized as follows. A UDP packet is transmitted to request data from the other end-point (a GET on a device URL). Next, a response packet including the requested information (e.g., the temperature value of a sensor) is sent back. Note that a packet of data can also be pushed to a device – a POST to its URL.

CoAP and HTTP protocols have many similar features with the difference that CoAP is optimized for IoT and more specifically for M2M. CoAP has low overhead and is very simple to parse. It has proxy and caching capabilities and exchanges messages asynchronously.

As it is shown in Figure4.3, CoAP is made of two different layers: messages layer which is the lowest layer and deals with UDP, and request/response layer which manages request/response interactions.

A.  CoAP Messages Layer

The messages layer sits on top of the UDP and deals with exchanging messages between the IoT devices and the Internet. The message duplicates are detected using the unique IDs assigned to each message. CoAP provides its own reliability mechanism using confirmable messages and non-confirmable messages.

Confirmable message (CON)

A confirmable message (CON) is a reliable message where the client keeps sending the message using a default timeout and a back-off mechanism between retransmissions until an acknowledge message (ACK) with the same ID is received from the server. The ACK messages can carry useful payload data and themselves do not need separate ACKs.

In case the server has trouble processing the CON message (e.g., when the server cannot even provide an error response), it replies with an RST message (reset message) instead of the ACK signal.

Non-confirmable messages (NON)

When exchanging non-critical messages, unreliable NON messages can be used where the server does not acknowledge the arrival of the messages. Although the NON messages are not acknowledged, they are assigned message IDs to detect the message duplicates.

CoAP unlike MQTT does not provide an explicit QoS level model. However, the confirmable/non-confirmable message types can be mapped to the MQTT QoS semantics. Table4.3 shows CoAP and MQTT QoS mapping.

Table4.3. CoAP and MQTT QoS mapping

B.  CoAP Request/Response Layer

This layer can send the request using either a CON or NON message. In a scenario where the request is sent using a CON message, an ACK message containing the response or the error code is responded by the server provided that it can answer immediately. In such a communication scheme, the request and the response are matched using a token (this token is different than the message ID).

In case the server cannot respond right away, it sends an ACK message with no content as the response (empty response). Once the response is ready, a new CON message containing the response is transmitted to the client. After receiving the response, the client will acknowledge that. Note that the server’s response will also be a NON message, if the request sent by the client is a also a NON message.

C.  Multicast Group Communications

Group communications (one-to-many or multicast) which allow a single request to be sent to many devices are supported by CoAP.

D.  Observers

Observers reduce the need for constant polling by allowing a client to receive updates based on the state of the subject. In other words, the state of a smart object is dependent on the states of the other devices that it is connected to. An example of observers in action is when a motor reacts to a sensor measuring the rotational speed of a generator, by adjusting its speed accordingly. The observer client’s status keeps updating until either a CON message is unacknowledged or an explicit removal message is sent by the client, or the last notification is expired.

E.  CoAP Security

Same as HTTP, CoAP is also a plaintext protocol by default which means that to secure the communication, an additional encrypted protocol as a wrapper is required. In CoAP the data encryption is done by Datagram Transport Layer Security (DTLS) over UDP to secure and protect the information.

4.4 Advanced Message Queuing Protocol (AMQP)

AMQP is a lightweight application layer protocol optimized for higher security and reliability, easier provisioning and interoperability. AMQP protocol supports both publish/subscribe (such as MQTT) and request/response (such as CoAP) architectures. AMQP is a connection-oriented protocol (the client and the broker need to establish a connection before they transfer data) because it uses TCP as its transport protocol. AMQP is a considered a reliable protocol with two levels of QoS for message delivery including unsettle format (similar to MQTT QoS 0) and settle format (reliable similar to MQTT QoS 1).

4.5 Comparative analysis of IoT protocols

As it was seen in the previous sub-sections each IoT protocol has its own data transfer model. For example comparing the two IoT protocols we discussed earlier in this chapter, MQTT and CoAP use publish/subscribe and request/response models, respectively. In MQTT, a central broker is used to forward the received messages from the publisher to the subscribers (MQTT can support multiple publishers and subscribers for the same topic). On the other hand, similar to the HTTP, CoAP is basically a one-to-one protocol. MQTT is an event-oriented protocol (the client will publish a topic whenever there is an update) while CoAP is more suitable to transfer the state of the device. The main difference between CoAP and MQTT results from the MQTT requirement to run on the reliable, lossless, in-order, byte-stream delivery transport, which effectively mandates the TCP/IP stack. CoAP, on the other hand, runs on top of UDP and provides its own reliability mechanism using confirmable and non confirmable message types. Overall, MQTT/TCP traffic requires approximately 10x more transactions and the data overhead is approximately 100x higher compared to the CoAP/UDP traffic for the 5-byte payload. The characteristics of the IoT protocols have been summarized in Table 4.4 [7].

Table4.4. Characteristics of the IoT protocols

Depending on the messaging requirements of IoT use cases, different messaging protocols may be selected. This is why it is important to understand the strength and limitations of each protocol to determine their best-fit applications. Table4.5 presents an evaluation and relative analysis of the four widely accepted and used IoT protocols discussed in this chapter. This relative comparison in terms of message size, power consumption, bandwidth, reliability, security, and interoperability and so on, helps to gain insight into the pros and cons of each protocol [7].

When thinking about selecting MQTT and CoAP the following recommendations shall be taken into account:

• For IP data transmission over NB-IoT networksو CoAP protocol is the preferred.
• CoAP can provide from 224 to 1,157 percent higher payload efficiency compared to the TCP/IP-based MQTT. So, it offers supe­rior transmission performance for both network capacity and IoT solutions.
• MQTT over NB-IoT networks is not recommended due to very high overhead and increased number of data transactions.
• MQTT is appropriate for relatively long-standing MQTT/TCP/IP con­nections that transmit relatively high volumes of data (e.g., telematics appli­cations). For IoT solutions that require MQTT transport, other types of radio technologies, such as LTE, should be considered.

Table4.5. IoT application protocols: comparative analysis