How Smart MQTT Clients & Brokers Maximize Your Results

MQTT is the Envelope, But What About Contents Format?

MQTT enables clients to publish and subscribe to data in cloud or premise hosted brokers that manage the data. It is basically an envelope for data. Clients publish data to brokers, and any other clients that want data can subscribe to data, organized in what are called topics, in MQTT brokers. Any client can publish and subscribe to data to one or more brokers. Clients can be software applications such as HMI or SCADA systems but are often intelligent field devices.

However, MQTT alone does not define the organization of data in packets, known as the payload. Each device or software supplier can define their own format for the data in the envelope as shown in the example at the right. This is typically done in a JSON format which is a form of structured text file containing name/value pairs. Although there is a draft IETF specification for JSON format standardization, there are still limitations:

• Each supplier can define their own formats, which causes interoperability issues that have to be coordinated at integration time. 
• Most MQTT clients and brokers can only parse one data format per MQTT connection, so users can also experience vendor lock-in due to vendor-specific payload formats needing to all be the same. 

Possible solutions to these problems include:

• MQTT Sparkplug B.
• Smart clients & brokers will:
  • Support Sparkplug B and regular JSON format connections.
  • Parse multiple JSON formats in a single connection including support for the IETF JSON draft specification.

MQTT Sparkplug B extends basic MQTT with a standardized payload format for users, integrators, and suppliers to use to define models for interchanging data. All systems must still agree upon the actual data model, but the interoperability risks are lower than just basic MQTT, enabling expanded usage of MQTT. 

For systems that are not using Sparkplug B and where there is a mixture of device types, each with their own JSON payload format, a key to interoperability is having Smart MQTT client/clients that can parse multiple JSON formats on a single connection to the broker, thus enabling subscribing to data coming from multiple devices. A Smart MQTT broker can also receive data from a Smart MQTT client able to publish multiple formats in one connection, then enabling that broker to serve that data out to other systems.

The Cogent DataHub software provides SmartMQTT client and broker functionality that can handle simultaneous Sparkplug B and plain MQTT connections, and parse multiple JSON formats over the same connection using it's advanced parsing functionality.

Integrating with Other Data Exchange Standards 

The realities of industrial operations are that there is no single communications protocol. Operations have PLCs, drives, control systems, DCS and more, that all speak their own protocols over serial and Ethernet connections.  The OPC Classic and now OPC Unified Architecture (UA) standards have become the bridge from device and vendor-specific serial and Ethernet protocols to a standard interchange method with HMI, SCADA, MES, Historians and more.  The OPC standards also integrate alarms and event data, which may need to be considered.  Effective IIoT and digital transformation initiatives must consider the need to exchange data across standards.

With standard MQTT clients and brokers, integration with OPC Classic, OPC UA and other systems and standards often:

• Requires the use of separate products that must be licensed and configured, adding cost, complexity, and risk.
• Cannot automatically create OPC tags for integration to other systems.

A Smart MQTT client and broker reduces risk, complexity and integration cost by providing:

• Native OPC Classic and OPC UA client and server interfaces that require minimal to no configuration to pass data between MQTT and these standards.
• Automatic creation of tags from Sparkplug B topics.
• Integrated ability to connect to databases, Modbus TCP, historians and more.

When using Cogent DataHub, users have the ability to turn on native, built-in OPC Classic and OPC UA connections as well as the others described above including alarms & events.

If DataHub does not have the integration method needed, Software Toolbox provides a range of OPC solutions for integrating things like OPC UA Pub/Sub, OPC UA Information Model Companion Specifications, SAP & other ERP systems, as well as tools for custom application developers to integrate into DataHub and OPC standards. 

Beyond Just the Messages

As powerful as MQTT is, when using information shared for decisions that can affect control operations, there are several areas you must consider, and we'll cover just a few of them here.  

Natively, MQTT supports what is called QoS or Quality of Service with 3 levels. QoS really just addresses delivery. 

• QoS 0 is a fire-and-forget type of transmission with no guarantee of delivery.
• QoS 1 has a simple retry, and will keep trying until the message is delivered, which can result in multiple deliveries of the same data.
• QoS 2 ensures the message will only arrive once.

This may sound good, but there are other things that can still go wrong that QoS does not address, which only Smart MQTT Clients and Brokers address.

Message Order

When clients publish to brokers, the broker's job is to get the data back out to subscribers as quickly as possible. It does not know what is in the message. There is no guarantee that data will arrive at subscribers in the order that it was received from publishers, which can result in inconsistent data. When making correct, informed control decisions relies on the order of data receipt, a Smart MQTT Broker will empower retaining that order. 

Data Quality Propagation

If you have multiple MQTT brokers chained together passing data between levels within a business or across a large application, QoS does not pass through from one regular broker to another. Smart MQTT brokers are able to handle moving data from one broker to another and assure delivery.

More importantly, when integrated with other automation systems, standard MQTT brokers cannot indicate anything about the quality of the connections to things such as OPC servers and their underlying hardware.  

Smart MQTT clients can use quality data from OPC and share it in the message format if your chosen format has a place for it. Smart MQTT Brokers will mark data as Not Connected when an MQTT client disconnects, so that subscribers can know that about the data.

Data Sampling

MQTT brokers are not unlimited in the amount of data they can receive. This can be for technical reasons, but also commercial if there are charges for how much data you publish to a paid cloud service you publish too.  Standard MQTT clients publishing process data that is changing at high rates, can quickly amass large charges, or create loading issues that affect the performance of the cloud service. 

The DataHub Smart MQTT Client can let users specify the limits of the broker in terms of subscriptions, delays between messages, maximum publishes per second, message length, and even topic name lengths.

The DataHub will then use your per-connection, per-broker settings, to manage sending only within the broker limits. 

Security

A typical MQTT broker just moves envelopes. If you need to restrict a topic, the read and write restrictions only occur at the client, not the broker. Without using a Smart MQTT broker, there is no way to restrict topics to be read only.  A Smart Broker will provide intelligence to be able to better secure data than standard MQTT brokers. 

There are numerous other details beyond these, but the use of a Smart MQTT client and Smart MQTT broker will provide reliable ways to address them. We encourage users to learn more about the DataHub and its capabilities, and consult with our DataHub experts to discuss applications in the planning and design phase, and leverage our DataHub free trial software to test scenarios.

When Data Loss Matters

When does data loss NOT matter is possibly the better question! MQTT is used in many applications where the device or client to broker connections can have problems. Often the connections are wireless and pass through multiple points of failure. 

For this reason, if your application has a low tolerance for data loss, you need MQTT tools that understand that. The MQTT client needs to be able to store data locally and retransmit or forward it when the broker connection is available again. As discussed earlier, message order matters, so your client also must be able to maintain that order. 

The DataHub Smart MQTT client has native store-and-forward capability that can be turned on with a license and put to work. For users that do not have data loss concerns, they can just use the Smart MQTT client standard capabilities.

Another consideration is if you need redundancy in your brokers, while still dealing with all the other considerations. Standard MQTT clients and brokers are not able to handle redundancy and deal with the other considerations at the same time.  The DataHub Smart Client and Broker can handle the challenges of redundancy at the MQTT level but also with redundancy in other integrated systems such as with OPC clients and servers. 

Conclusions

The realities of using MQTT in industrial applications require attention to detail. Users & integrators need solutions that understand those details and partners that are able to provide technical expertise in all phases of the project - from planning and design to implementation. 

Software Toolbox provides that expertise bundled with off-the-shelf solutions such as the Cogent DataHub and their suite of standards-based industrial integration software. 

You can download a free trial of the DataHub below, and contact our experts for help in all phases of your application. 

The DataHub free trial version:

• Is fully functional without any artificial tag count or functionality limitations.
• Runs for 1 hour at a time with longer evaluation time licenses available upon request.
• Can be rapidly configured using our MQTT configuration How-To videos.