Skip to content

MQTTController

MQTTController is the Reactor interface to the world of MQTT. It requires a MQTT broker running somewhere on the network (MQTTController is not itself a broker, it's a client). The MQTT broker must be fully MQTT 3.1.1 compliant (or higher). The reference broker used for development and testing of MQTTController is Eclipse Mosquitto. Most Linux distributions can install this from their package managers, and docker containers are also readily available.

openLuup MQTT Broker

Users choosing the MQTT broker built in to openLuup must be running at least openLuup version 2022.12.2. Earlier versions are not supported. Additionally, this broker does not support subscriptions at QoS levels 1 and 2, and does not persist retained topics across restarts of openLuup, so it's not fully 3.1.1 compliant. Still, it has basic functionality and is convenient if you're already running openLuup, just be aware that retained messages can be lost during reboots/recoveries, and this can lead to entity states being inconsistent with their devices/clients and thus incorrect automation execution. The init configuration key described in Additional Entity Configuration below, when used with a device/client inquiry topic to restore synchronization of data, is a possible work-around.

There are three functions that MQTTController performs at the highest level of perspective:

  1. It maps devices that send and receive MQTT events into Reactor entities. For example, a Shelly1 relay device or an ESP8266 flashed with Tasmota can connect to an MQTT broker and exchange messages. Reactor connected to the same broker will "hear" status updates published by connected devices, and can update attributes on a Reactor entity accordingly (e.g. set the temperature_sensor.value attribute to the value of the temperature published), and actions performed on the Reactor entity can be turned into command messages to the device.
  2. It can "echo" any entity in Reactor to the MQTT universe, effectively acting as a gateway so that all entities known to Reactor are published.
  3. Commands can be issued to Reactor using MQTT to run actions on any Reactor entity.

MQTTController can also serve as a local gateway for MQTT messages for other Reactor controllers (that is, other controllers can ask MQTTController to subscribe to topics and exchange datagrams on their behalf).

Installation

MQTTController is not included in base Reactor distributions. It must be installed separately by downloading the package from the extras subdirectory of the Reactor download server.

Once the package file (a ZIP or Gzip'd tar archive) is downloaded:

  1. Create, if it does not already exist, a directory called ext in your Reactor install directory (so it should be at the same level as config, storage, etc.).

    cd /path/to/reactor
    mkdir ext
    

    If you are running Reactor in a docker container, the ext directory should be created in the data directory, where your config and storage directories live as indicated above.

  2. Change directory into the new ext directory:

    cd ext
    
  3. Unpack the archive (use whichever command is correct for the type of archive downloaded):

    tar xzvf /path/to/MQTTController-XXXXX.tar.gz    # or...
    unzip /path/to/MQTTController-XXXXX.zip
    
  4. Run the install script. If you are using a "bare metal" install (not a docker container):

    cd MQTTController
    ./install.sh
    

    If you are running Reactor in a docker container, we will open a container shell in which to do the install (the Reactor container must be up and running):

    docker exec -it <container-name> /bin/sh
    cd /var/reactor/ext/MQTTController
    ./install.sh
    exit
    

From here, proceed to Basic Configuration below.

Basic Configuration

In order to use MQTTController at all, you have to add an entry for it to the controllers section of your reactor.yaml file.

controllers:
  # Your existing controllers will be below the above line.
  # Add the following after the last "- id" line in this
  # section.
  - id: mqtt
    name: MQTT
    enabled: true
    implementation: MQTTController
    config:
      # Replace IP with that of your MQTT broker below
      source: "mqtt://127.0.0.1:1883/"

Restart Reactor to make the changes take effect. After that, you should be able to refresh the UI, go the Entities list, clear any existing filters, and choose "MQTT" from the controllers filter selector. That should then show you two entities: the MQTT controller system entity, and its default group entity. If you don't see this, check the log for errors.

If you need to authenticate with your MQTT broker, you can supply username and password fields at the same level as source.

Configuring Devices

Devices are configured within an entities subsection of the config section for MQTTContoller.

MQTTController uses a data-driven model to map published messages to attributes on Reactor entities, and entity actions to published commands. There are two ways to map a device:

  1. If an existing preconfigured template will work for your device, you provide the topic that identifies your device, and tell MQTTController what template to apply to it.
  2. If there is no existing template that works for your device, you create one by telling MQTTController what Reactor capabilities the device has, and how to map published messages with state/telemetry updates into the attributes of those capabilities. You will also tell Reactor how to take actions in those capabilities and map them to topics to be published to make the device do what you need it to do.

The preconfigured templates available are listed at the bottom of this page. They are defined in mqtt_devices.yaml, in the templates section. Do not modify this file. This is a system file. There is a separate mechanism for you to define your own templates without modifying the system file.

Easy Device Configuration - Using Existing Template

As an example, let's say we have a Shelly1 configured with the topic shelly1_43DFD2 that is configured to operate as a single relay to operate a stairway light. To create an entity for this device, we can use the shelly_relay template. The template supports multi-relay devices, but the Shelly1 only has a single relay on channel 0:

...other stuff
    config: "mqtt://127.0.0.1:1883/"
    entities:
      shelly1_stairway:
        name: "Stairway Light"
        topic: shelly1_43DFD2
        uses_template: shelly_relay
        channel: 0

After applying this configuration, we restart Reactor, and then refresh the UI. We should then be able to find the Stairway Light entity in the Entities list. You should be able to use the power_switch.on and .off actions to turn it on and off, and observe that the state correctly tracks that of the device, even when operated from the Shelly mobile app or any other means.

!!! tip Use Retained Messages! It is strongly recommended that you configure your devices to use retained MQTT messages. This ensures that Reactor (and any other MQTT-connected apps you may be using, like Hass) get the latest state from the MQTT broker automatically at startup. Some devices, notably Tasmota, do not have MQTT retain on by default (a poor choice for IoT devices, IMO). On Tasmota devices, you can turn it on by using the PowerRetain on, SwitchRetain on, SensorRetain on and StateRetain on commands in the Tasmota console. Check with your device's documentation for the default state of the retain flag on messages it publishes, and how to turn it on if the default is off.

Detailed Device Configuration

If an existing template doesn't supply the behavior you need, you can configure the entity directly. Later, if you wish, you can turn your configuration into a reusable template, so if you have many similar devices, the configuration can be shared between then.

To get MQTTController to create and maintain an entity for your MQTT-enabled device, you need to create a mapping configuration that tells MQTTController:

  • What type of entity to create;
  • What capabilities it will have;
  • How to turn state/status topics received for the node into attribute values on the entity;
  • How to turn action requests for the entity into published MQTT messages.

To illustrate how this is done, let's implement a configuration for a Tasmota in generic/relay configuration (there's a template for this, but we're doing it explicitly here to show how that's done). Our Tasmota device is configured for a single relay, basically a switch, so we're going to map it to a Switch type entity with the two capabilities that most switches have in Reactor: power_switch and toggle.

We'll begin with our basic entity configuration. First, we need to assign a unique ID for the entity. To make it easy, we're going to use the Tasmota node's ID:

  - id: mqtt
    config:
      source: "mqtt://127.0.0.1:1883/"
      entities:
        "tasmota_FFFFFF_relay":  # this is your new entity's ID
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"

Rules for Assigning Entity IDs

Entity IDs are identifiers, not names, and as such, the rules for valid IDs are strict: they must be between 1 and 64 characters in length; they may contain only upper- and lower-case letters A-Z, digits 0-9, and underscore; the ID must be unique on that MQTTController instance (it does not need to be unique system-wide). It should also never change, as rules and reactions will use a stored copy of the entity ID to find your entity, so if you change it, those rules and reactions will not find your entity and throw an error until you update them as well.

Now we can get into configuring what the entity can do:

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
          capabilities:
            - power_switch
            - toggle
          primary_attribute: power_switch.state

OK, the above additions tell MQTTController that we're going to create a Switch type entity, and it will have capabilities power_switch and toggle. Its primary attribute will be power_switch.state.

Info

Don't worry if you don't recognize the Switch entity type, or know what you should put there. This value is used by the Dashboard to guess a default configuration for tiles, but since the Dashboard isn't really the focus of work right now, you can ignore this value. In fact, you can leave it out altogether; it won't hurt anything.

The power_switch capability is the only one of the two being assigned that has any attributes, and its only attribute is that state attribute being assigned as the primary. We need to get the value out of status update topics we receive and into that attribute. Tasmota sends topics of the form stat/nodeID/POWER for the status of a single relay device. When the relay is turned on, the payload is the string on, and when it's turned off, the payload is off. We'll create an events section, and put a handler in for that topic.

          ...
          primary_attribute: power_switch.state
          events:
            "stat/%topic%/POWER":
              "power_switch.state":
                expr: "bool(payload)"

The first thing to notice is the %topic% string in the event topic. This is a substitution: MQTTController will put the string in the configuration value topic for this entity into that string. We could also have just hard coded it (that is, use stat/tasmota_FFFFFF/POWER), but if we want to make a template out of this later, it's better to use the substitution.

So having told MQTTController what topic this entity is interested in, we then tell it what attributes that topic can effect. This topic only affects one attribute, power_switch.state, and we use an expression to convert the payload string (on or off as stated above) to the boolean type that is required for power_switch.state. The expression here is trivially easy because the function bool() recognizes and converts the string on, yes and true to true, and off, no and false to false, and is not case-sensitive.

If that topic could affect more attributes, we would list them using the same structure as that described for power_switch.state.

Likewise, if there were other topics that could affect attributes (including other topics that would affect power_switch.state), we could just list them in the events section and structure them as we have done so far. MQTTController will make sure every topic gets processed by every entity that is interested in it, and every attribute that can be affected by it.

Note that not every topic/payload sent may contain the data necessary for every attribute. To avoid errors from expressions and/or accidentally setting your attribute values to null in these cases, you can add if_expr to test if the payload contains the data needed to update the attribute. The expression should result in boolean true or false; if false, the attribute will not be evaluated further and will not be modified from its current value. Here's what that looks like for an additional Tasmota topic:

            "tele/%topic%/STATE":
              "power_switch.state":
                json_payload: true
                if_expr: '! isnull( payload?.POWER )'
                expr: 'upper( payload.POWER ) == "ON"'

Also note that we've set json_payload to true here. This tells MQTTController to parse the payload as JSON and make it available as a real object, rather than a long text string, to your expression(s) for that attribute. If the payload is not parsable as JSON, the resulting object will be null, so notice that we've guarded the if_expr for this case using a coalescing member access operator (?.). The result of this if_expr is therefore false if either (a) the payload isn't JSON, or (b) it's a JSON payload but the resulting object does not contain a member named POWER.

Now let's move on to actions. In the actions section, we list each capability that has actions, and within each of those, we list each action and its implementation. Let's start with power_switch.on and power_switch.off. These simple actions take no parameters, so their implementation is very simple:

          events:
             ...
          actions:
            power_switch:
              "on":
                topic: "cmnd/%topic%/Power"
                payload: "on"
              "off":
                topic: "cmnd/%topic%/Power"
                payload: "off"

Under each action, the required value topic sets the topic to be published to implement the action. If the topic has a payload, then the payload value can be added. The payload shown here for both actions is just a string, because the payload for these commands is a string, and in this case, it's a fixed string (it is always on for the on action and off for the off action. The power_switch.set action makes things a little more complex, because it takes a boolean parameter named state to determine what to do with the light/switch/relay. Here's that implementation:

          actions:
            power_switch:
              ...
              set:
                topic: "cmnd/%topic%/Power"
                payload:
                  expr: "parameters.state ? 'on' : 'off'"
                  type: raw

In this implementation, the topic is the same, but the payload is more dynamic. The paylaod value is an object with two keys. The expr key/value is an expression that figures out whether to return the string on or off based on the value of the state parameter. The type key/value of raw tells MQTTController that the string returned by the expression should not receive any translation. So the payload thus becomes the string on or off sent with this topic to implement this action.

Some topics have more complex payloads. It's not usual for a topic to have a JSON payload, for example. In this case, the expr value may return an object, and if the type is set to json, MQTTController will JSONify the expression result to a string that is then sent as the payload.

If your action wants to set the MQTT qos level or retain flag, just provide those using keys of the same name. The value of qos should be an integer, and retain should be boolean.

Since we've done this in chunks, here's the whole configuration all together:

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
          capabilities:
            - power_switch
            - toggle
          primary_attribute: power_switch.state
          events:
            "state/%topic%/POWER":
              "power_switch.state":
                expr: 'bool(payload)'
            "tele/%topic%/STATE":
              "power_switch.state":
                json_payload: true
                if_expr: '! isnull( payload?.POWER )'
                expr: 'upper( payload.POWER ) == "ON"'
          actions:
            power_switch:
              "on":
                topic: "cmnd/%topic%/Power"
                payload: "on"
              "off":
                topic: "cmnd/%topic%/Power"
                payload: "off"
              set:
                topic: "cmnd/%topic%/Power"
                payload:
                  expr: "parameters.state ? 'on' : 'off'"
                  type: raw

Notice anything missing? We haven't defined the toggle.toggle action. The toggle capability has no attributes and just that one action, but we haven't defined what that action does. As it turns out, MQTTController can supply a default implementation for toggle.toggle, so we don't need to define it.

Additional Entity Configuration

The lwt key may be specified to identify the topic sent by your device as a Last Will and Testament (e.g. lwt: tele/%topic%/LWT is common for Tasmota). When this topic is received, MQTTController will mark the entity down by setting x_mqtt_device.online to false. The specific conditions under which the LWT is sent are device-dependent; consult your device and broker documentation for more details. Also, note that this default handling does not examine the payload; it assumes that receiving the LWT with any payload is an indication that the device is down (or going down). If this implementation isn't appropriate for your device, don't use the lwt key and handle the device's LWT topic and the x_mqtt_device.online attribute explicitly in your configuration.

The query key can be used to provide a topic that causes the device to report its current state. A payload is not allowed, only a topic (string). When this optional key is provided, the x_mqtt_device.poll action is enabled and will cause the query topic to be sent to prompt a device update.

The init key may be used to send initialization strings to a device. These strings will be sent every time Reactor connects to the MQTT broker. This is useful for ensuring that the remote device/client is in a known state. It can also be used to query the current state of the device/client to ensure the Reactor entity is in sync with it.

The init value may be:

  1. A string, in which case the topic (with no payload) is the only topic sent at initialization;
  2. An array, containing any number and mix of:
    • string, which are topics to be sent (with no payload);
    • object, containing the required key topic and the optional keys payload, qos and retain.

Here's an example of how these are formatted in the configuration (when needed):

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
          capabilities:
            - power_switch
            - toggle
          primary_attribute: power_switch.state
          query: "cmnd/%topic%/Power"
          init:
            - "cmnd/%topic%/someAction"
            - topic: "cmnd/%topic%/anotherAction"
              payload:
                level: 50
                rate: 3
              type: json
              qos: 0
              retain: false

Controlling Attribute Evaluation Order

With the structure of event configuration discussed so far, the order in which attributes are evaluated and updated is not deterministic. That is, if an event is configured to update two or more attributes, there is no defined order by which the attributes are processed, and their order in your configuration is of no consequence.

If the order of processing is important, you can use the array form of event configuration. In this form, the data under the event's topic is an array of objects, each containing the attribute key to specify the attribute name. The remainder of the object is the same.

Let's look at a specific example, highlighting the differences in the format and why we might care:

    events:
      "%topic%/power:
        "voltage_sensor.value":
          expr: "float( payload.load.voltage )"
        "power_sensor.value":
          expr: "float( payload.load.power )"
        "current_sensor.value":
          expr: "entity.attributes.power_sensor.value / entity.attributes.voltage_sensor.value"

In this case, we have a hypothetical solar inverter/controller that is sending topics to give us instantaneous power output and line voltage, but it doesn't report amperes, so we are computing the amperes (ignoring power factor, etc.) by simply dividing power (in watts) by voltage.

The problem here is that there is no guarantee, in this typical configuration form, that when the topic is received, the three attributes are evaluated and updated in the order shown. The current_sensor.value attribute is relying on the entity's newly-updated voltage and power values, but if this attribute happens to be evaluated first or second, those entity attributes may not yet have been updated, and the computed current will be incorrect.

To fix this, we can reformat the configuration like this:

    events:
      "%topic%/power:
        - attribute: "voltage_sensor.value"
          expr: "float( payload.load.voltage )"
        - attribute: "power_sensor.value"
          expr: "float( payload.load.power )"
        - attribute: "current_sensor.value"
          expr: "entity.attributes.power_sensor.value / entity.attributes.voltage_sensor.value"

The key change made for each entry is the addition of - attribute: before the attribute name. This does two things: it refines the entry as an array element, and it makes the attribute name a key/value pair in the object that is the array element (expr is another key in each respective object).

In this array-based structure, the order in which the attributes are processed is guaranteed to be the order in which they appear in the configuration: voltage will be updated first, followed by power/watts, and finally current/amps.

Converting Your Configuration into a Template

In the previous section, we built the device configuration directly into our per-device configuration. If we had several such devices, though, it would be verbose and more difficult to maintain all of them with each device having a complete copy of all those lines. We can simplify our configuration by converting our device configuration into a template.

To begin, create a directory in your config subdirectory called mqtt_templates. Next, we'll assign an identifier (name) to our template. We'll call it my_tasmota_relay, and create a YAML file with that basename (e.g. my_tasmota_relay.yaml) in the mqtt_templates subdirectory.

# Optional description of what your template does (this is a comment)
my_tasmota_relay:

Now, we can basically copy the device configuration we previously created, leaving behind the ID, name and topic, since these are values that will later be passed to the template from entity configuration (that is, we don't want the template to have fixed/overriding values for these, so that the template can be reused to make multiple entities).

Passing data from an entity configuration to the template is straightforward. In any %keyword%-form substitution, the replacement string will be the value from the key of the same name in the entity's configuration. As a trivial example, if you add flavor: raspberry to your entity's configuration, then in any topic strings in your template you could include %flavor% and that would be replaced in runtime with the word raspberry (e.g. topic /icecream/%flavor%/rating becomes /icecream/raspberry/rating in use). This allows you to fully parameterize your templates. In addition, as of version 22282, if your substitution is of the form %flavor:chocolate%, then if flavor is not defined in the entity's configuration, the substitution will use the word chocolate, so you can establish defaults. For an example of this kind of parameterization in operation, look at the implementation of the "real" shelly_relay template in MQTTController's mqtt_devices.yaml file, which uses channel and unit to handle multi-relay devices.

You can also access configuration data in your template's expressions. The config object is defined in expression context, and allows you to access the elements of the entity's configuration. Building on the example in the previous paragraph, your expression could refer to config.flavor to access the configured value of the flavor configuration value.

If your template uses required substitutions or expression-context values, it is recommended that you add a requires: [] entry to the template configuration (before events and at the same indent level), specifying as the array contents the names of all values required by the template (e.g. required: [topic, unit]). If a user of your template omits one of these required values from their configuration, a detailed, helpful alert will be displayed and logged. The array form must be used even if there is only one required value.

Your completed template file will look something like this:

# Optional description of what your template does (this is a comment)
my_tasmota_relay:
  type: Switch
  capabilities:
    - power_switch
    - toggle
  primary_attribute: power_switch.state
  requires: [topic]
  lwt: "tele/%topic%/LWT"
  events:
    "state/%topic%/POWER":
      "power_switch.state":
        expr: "bool(payload)"
    "tele/%topic%/STATE":
      "power_switch.state":
        json_payload: true
        if_expr: '! isnull( payload?.POWER )'
        expr: 'upper( payload.POWER ) == "ON"'
  actions:
    power_switch:
      "on":
        topic: "cmnd/%topic%/Power"
        payload: "on"
      "off":
        topic: "cmnd/%topic%/Power"
        payload: "off"
      set:
        topic: "cmnd/%topic%/Power"
        payload:
          expr: "parameters.state ? 'on' : 'off'"
          type: raw

In your device configuration, you can refer to your custom template with the uses_template key. Here's our revised device configuration, with an additional entity of the same type added for another Tasmota relay node, using the template for both:

  - id: mqtt
    ...
    config:
      source: "mqtt://127.0.0.1:1883/"
      entities:
        "tasmota_FFFFFFF_relay":
          name: "Tasmota Relay 1"
          topic: "tasmota_FFFFFF"
          uses_template: "my_tasmota_relay"
        "tasmota_BEEFEE_relay":
          name: "Tasmota Relay 2"
          topic: "tasmota_BEEFEE"
          uses_template: "my_tasmota_relay"

Echo Capability - Publishing Reactor Entities

MQTTController can, if configured, publish messages for every state change of a set of entities you choose (or all entities), rules, and global variable changes. In this way, all entities known to Reactor may become MQTT-enabled, regardless of their source. MQTTController also knows how to receive topics to perform actions on these entities.

Info

I'm not sure what to call this function, so for now I'm just calling it "echo" capability. Gateway...? Relay...? Great and Powerful Oz? Any ideas?

Enable echoing by defining an echo section in your MQTTController configuration:

  - id: mqtt
    ...
    config:
      source: "mqtt://..."
      entities:
        ...
      echo:
        #include_entities: []
        #exclude_entities: []
        #include_groups: []
        #exclude_groups: []
        #include_capabilities: []
        #exclude_capabilities: []
        primary_attributes: true
        qos: 0
        retain: false

Note

Some configuration keys have been commented out (#). As shown above, the defaults will be used for all of these keys. Uncommenting some of these keys immediately changes behavior, so be sure you read the descriptions of each below before doing so.

The include_ and exclude_ filters determine what data is echoed. They are arrays, and you can either use the compact array notation (['string1','string2']) or the multi-line list form (preferred for lists of more than 2 items).

    list_name:
      - "list element 1"
      - "list element 2"
      - "list element 3"

In MQTTController, the rules for include/exclude filter lists are as follows:

  1. If an include list is not defined (i.e. commented out or does not exist at all in the config), everything passes (jump to step 3).
  2. If the include list is defined, then only those items matching any element pass.
  3. If an exclude list is not defined, nothing is excluded (except in capabilities, see below).
  4. If the exclude list is defined, then those items matching any element are excluded.

Beware Empty Arrays

The meaning of an empty array ([]) is different from that of being commented out or undefined. It is interpreted to mean nothing; therefore, in include lists, an empty array would mean include nothing, so be particularly careful there.

Filtering is done in the order: entities, groups, capabilities. Within each type, the include list is processed before the exclude list. So the specific order of filtering is: include_entities, exclude_entities, include_groups, exclude_groups, include_capabilities, exclude_capabilities.

The include_entities key is an array containing canonical ID patterns to match. The patterns are defined as follows:

  • Literal canonical IDs, like hubitat>100, to refer to a single entity;
  • Wildcard to match all entities on a particular controller: hubitat>* matches all entities from the controller with ID hubitat;
  • Regular expressions: /pattern-regexp/optional-flags

For example (using alternate array syntax for improved readability):

    echo:
      include_entities:
        - "vera>*"
        - "hass>*"

This would match all entities from the controllers with IDs vera and hass. If included_entities is not specified (i.e. commented out or not present in the config), then all entities from all controllers are included.

After inclusion, exclude_entities is processed in the same manner as include_entities, except that any entity that matches will be discarded.

The optional include_groups and exclude_groups arrays give further control over the entities echoed. The elements of these arrays are canonical IDs of groups. If an entity is a member of any of the listed include_groups, it will be allowed to echo. If it is a member of any of the listed exclude_groups, it will not echo. Dynamic changes in group membership will affect these filters dynamically. Group filters require MQTTController version 21343 or higher.

From the final set of entities that have passed filtering so far, the include_capabilities and exclude_capabilies lists will determine what capabilities are published as topics. If include_capabilites is not defined, all capabilities will be published (except those excluded). If exclude_capabilities is not defined, by default the extended (x_) capabilities will be excluded. If you wish to prevent this default exclusion and include all capabilities including the extended, simply set exclude_capabilities to an empty array (i.e. exclude_capabilities: [], meaning nothing is excluded).

Regular expression patterns are supported in include_capabilities and exclude_capabilities values, using the form /regexp/optional-flags (e.g. /^x_hass\b/i would be a case-insensitive match for any capability starting with x_hass).

The keys qos and retain determine the QoS and retentation of the state messages published. By default, these messages are sent at QoS 0 and no retention, but you may change these behaviors by providing different values as defined by MQTT.

If you do not needs rules or global variables published in your configuration, you can set publish_rule_states and/or publish_global_variables, respectively, to false to disable that function.

Working with Entity Topics

When a capability for an entity is published, the topic will be reactor/<mqtt-id>/<ctrl-id>/<entity-id>/state/<capability-name>, and the payload is a JSON object containing the attribute key/value pairs of that capability for that entity. The mqtt-id is the ID of the MQTTController, the entity-id is the ID of the entity that is the subject of the message, and the ctrl-id is the controller ID to which that entity belongs.

reactor/mqtt/vera/device_10/state/power_switch {"state":true}

The setting of primary_attributes in the echo section determines whether a generic value topic is sent for the primary attribute of the entity. If this is true, you will see topics of the form reactor/<mqtt-id>/<ctrl-id>/<entity-id>/value with a payload string containing the primary attribute value. For a switch, the primary attribute of which is typically power_switch.state, one might expect:

reactor/mqtt/hass/switch_living_room/value true

For some users, it may be convenient for the entity name to also be part of the topic on announced attribute changes. MQTTController allows you to add the entity_identifier configuration key to the echo section with a value of id (the default) or combined. If combined or id/name is specified, the entity name will be added as an additional level of the published topic, such as reactor/mqtt/vera/device_10/Living Room Lights/state/power_switch. If name/id is specified, the name will precede the ID. Note that while these options add a level to the published topics for attribute changes, they do not change any other topic structures, such as those for performing actions (see below).

MQTTController can handle specific received topics and turn them into actions on Reactor entities. To perform an action on an entity, publish a topic in this form:

reactor/<mqtt-id>/<ctrl-id>/<entity-id>/perform/<action-capability>/<action-name>

If the action takes parameters, encode the parameter key/value pairs as a JSON payload. Here's an action to turn on our fictional living room light:

reactor/mqtt/hass/switch_living_room/perform/power_switch/on    (no payload)
reactor/mqtt/hass/switch_living_room/perform/power_switch/set {"state":true}

Both of the above topics accomplish the same task; the first requires no parameters/payload, while the second requires a "state" parameter. Here's a more complex payload example, setting color on an RGB bulb:

reactor/mqtt/vera/device_188/perform/rgb_color/set_rgb {"red":255,"green":128,blue:16}

MQTT Actions Always Available

Reactor will receive entity action messages and process them whether the echo function is enabled or not, and will process actions for any entity/capability, whether it matches echo filtering or not.

MQTTController will send a full update for an entity if it receives the following message:

reactor/<mqtt-id>/<ctrl-id>/<entity-id>/query

Working with Rule Topics

If rule publishing is enabled (which it is by default when echo is enabled), changes to the set/reset state of all rules will be published in topics of the following form:

reactor/mqtt/Rules/rule-id/state

The payload for this topic will be boolean true if the rule has set, and false if it has reset.

To force the publication of a rule state message at any time, you can send:

reactor/mqtt/rules/rule-id/query

You cannot control the state of rules via MQTT. Rule state is driven exclusively by the result of its conditions.

To help reduce traffic, you can disable publishing of rule state topics when echo is enabled by adding publish_rule_states: false to your MQTTController configuration.

Working with Global Variables

If global variable publishing is enabled (the default when echo is enabled), a value change to any global variable will cause a topic of the following form to be published:

reactor/mqtt/Expr/expression-name/value

The payload for this message is the (JSON) expression value. Be aware that primitive numbers and strings, when converted to JSON, are not surrounded by {} as many seem to expect for JSON. A number converted to JSON is simply a string containing the number (e.g. 123), and a boolean will be the string true or false. A string value will be quoted (e.g. "I am a valid JSON result"). Arrays and objects will, of course, have the familar {} envelope.

You can set the value of a global variable, if it is expression-less, by sending the value as JSON using same semantics described in the preceding paragraph on a topic of the form:

reactor/mqtt/Expr/expression-name/set

To query the value of a global variable any time, send:

reactor/mqtt/Expr/expression-name/query

The echo feature does not need to be enabled to set or query a global variable as shown above.

To help reduce traffic, you can disable publishing of global variable topics when echo is enabled by specifying publish_global_variables: false to MQTTController your configuration.

Additional MQTTController Behaviors

MQTTController will respond with a reactor/<mqtt-id>/pong message when it receives the topic reactor/<mqtt-id>/ping. If a payload is received, it is echoed back; otherwise, the current Reactor host date/time is returned as an Epoch time in milliseconds.

Troubleshooting and Debugging

To see what messages MQTTController is receiving, you can enable topic logging in the config section of its controller configuration. A list of topics or partial topics to match can be provided, and the matching topics will be logged to <id>-topics.log, where <id> is the ID of your MQTTController.

The following will log all topics received from Owntracks and BlueIris, for example (lines before log_topics are shown for context and indenting reference and should not be copied:

    implementation: MQTTController
    enabled: true
    config:
      source: "mqtt://127.0.0.1:1883/"
      log_topics:
        - "owntracks/"
        - "blueiris/"

If you're not sure what topic you're looking for and you want to log all topics, the following will match and log all topics received by MQTTController:

      log_topics:
        - ".*"

Fast- and Ever-Growing Log File Danger

Topic logging is meant to be used briefly, not to be left on. The topic log file will grow forever and eventually fill up the disk. It is not managed, pruned, or truncated on restarts. For this reason, do not leave any topic logging on any longer than you need it. Turn it off and remove the log file from your logs directory to recover and preserve your disk space, and prevent the system instability and errors that an "out of space" condition can cause.

List of Pre-defined Templates

The following templates are currently defined. If you work out a custom configuration for a commonly-used device, please consider contributing your template!

Template Name Use/Description Parameters
shelly_relay Creates an entity that maps one relay channel of a Shelly node to an entity. topic, channel
shelly_input Creates an entity that that maps one input channel of a Shelly node to an entity. topic, channel
shelly_temperature Creates an entity to contain the temperature sensor data from a Shelly device. topic
shelly_humidity Creates an entity to contain the humidity sensor data from a Shelly device. topic
shelly_motion Creates an entity to reflect the motion/no motion state of a Shelly motion sensor. topic
shelly_smoke Creates an entity to reflect the state of a Shelly smoke detector. topic
shelly_flood Creates an entity to reflect the dry/wet state of a Shelly flood sensor. topic
shelly_door_window Creates an entity to reflect the opened/closed state of a Shelly flood sensor.
shelly_battery Creates an entity to contain the battery level of a battery-operated Shelly device (e.g. a Shelly HT). topic
shelly_power Creates an entity with a Shelly device's power (watts) measurement. topic
shelly_tilt Creates an entity to contain the measurement of a Shelly device's tilt sensor. topic
shelly_vibration Creates an entity to contain the measurement of a Shelly device's tilt sensor. topic
tasmota_generic_relay Maps payload data and actions to the entity (capabilities switch_power and toggle); use optional unit to specify relay number if the device supports/reports more than one (default: blank/no unit). topic, unit
tasmota_detached_button Maps payload data for a detached switch input to the entity (capability button); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit). topic, unit
tasmota_detached_switch_mode0 Maps payload data for a detached switch in mode 0 (toggle/momentary; capability button); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit). topic, unit
tasmota_detached_switch_mode1 Maps payload data for a detached switch in mode 0 (on/off; capability binary_sensor); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit). topic, unit
tasmota_sensor_temperature_humidity Maps temperature and humidity values from payload into a single entity (capabilities temperature_sensor and humidity_sensor); use source to identify sensor subunit (default: Global). topic, source
tasmota_sensor_temperature Maps temperature (only) from payload into the entity (capability temperature_sensor); use source to identify sensor subunit (default: Global). topic, source
tasmota_sensor_humidity Maps humidity level (only) from payload into the entity (capability humidity_sensor); use source to identify sensor subunit (default: Global ). topic, source
tasmota_sensor_battery Maps payload battery data to the entity; requires source to identify the sensor subunit (default: Global). topic, source
tasmota_sensor_co2 Maps payload carbon dioxide level data to the entity; requires source to identify the sensor subunit (default: Global). topic, source
tasmota_sensor_dewpoint Maps payload dewpoint to the entity; requires source to identify the sensor subunit (default: Global). topic, source
tasmota_sensor_rssi Maps payload RSSI to the entity; requires source to identify the sensor subunit (default: Global). topic, source
tasmota_sensor_flora1 Mi Flora plant sensor for Temperature, Illuminance, Moisture and Fertility, use source to identify subunit (default: Global) topic, source
owntracks_in_region2 Creates a binary sensor the state of which is driven by an iPhone or Android device running OwnTracks. The sensor is true when the device is in the given region. The topic is the Device ID assigned in the OwnTracks application's configuration. The regionName is the name of a region defined in OwnTracks. All parameter values are case-sensitive (must match the case used in the OwnTracks app). topic, regionName, user

Community Contributions

Community users are encouraged to contribute templates for new devices. These may either be included in releases, or we'll link to them here if their work is large and growing. To install and use a community template, create a mqtt_templates subdirectory in your config directory, and place the template file in that subdirectory. You can then configure an entity using that template.

  1. More Shelly and Tasmota templates from dbochicchio/@therealdb: Github repository.

Using MQTT from Other Controllers

MQTTController has a mechanism that allows other controllers to subscribe to MQTT and send topics. This saves the other controller from having to create and manage a separate connection to the MQTT broker. While it creates a dependency on MQTTController, it is probably the case that a user using MQTT for your controller's device(s) is probably already using MQTT for other devices in their infrastructure, and it's already configured and running.

MQTTController does not require any special configuration for these tools to work. It doesn't even need to be configured beyond being enabled and having a correct source to find the broker. No events, entities, or echo need to be configured; MQTTController just needs to be up and running.

To subscribe to a topic, your Controller instance will need to do the following:

  1. Find the running instance of MQTTController. This is done by calling the getControllerByID(id) method of Structure, the instance of which you can get from the Controller base class by simply referring to this.getStructure(). The id will typically be mqtt for users that just follow the recommended installation instructions for MQTTController. You may want to make this a configurable string in your controller in case the user deviates from this recommendation, or has multiple MQTTController instances and wants to specify which to use. The following code snippet demonstrates all of this working together in a Controller subclass method:

    const mqttcontroller = this.getStructure().getControllerByID( this.config.mqtt_controller || 'mqtt' );
    
  2. Write a handler in your Controller subclass to receive your subscribed MQTT topics. You can have as many handlers as you wish: handle different topics with separate handlers, or handle all of your subscriptions in one handler; it's up to you. The handlers are methods of your Controller subclass that will receive arguments topic and payload when a topic matching your subscription arrives.

    handleMyTopic( topic, payload ) {
        // your implementation here
    }
    
  3. Request a subscription from the MQTTController instance by calling its extSubscribeTopic( this, topic, callback ) method. The first argument this is a reference to your controller instance (and typically the keyword this is exactly what is used in your code). The topic can include the typical + and # wildcards for subscribing to MQTT topics. The callback is a function reference or closure. When a matching topic is received by MQTTController, it will call the function with topic and payload arguments. To use a method of your subclass as the callback, use the JavaScript function bind() as shown below.

    // Using bind() to use a method of your controller as handler:
    mqttcontroller.extSubscribeTopic( this, "opensprinkler/#", this.handleMyTopic.bind( this ) );
    // Using a closure as handler:
    mqttcontroller.extSubscribeTopic( this, "tele/+/status", ( topic, payload ) => {
        /* Handling code here */
    });
    

    Note that it is possible to omit the callback argument. In this case, your controller's notify( event ) method will be called with a ext-mqtt-controller event. The topic and payload will be in the data of the event object.

To unsubscribe from a topic, call MQTTController's extUnsubscribeTopic( this, topic ) method. The topic must exactly match a topic string previously used in extTopicSubscribe(). If topic is omitted or null, all topics to which your controller subscribed are unsubscribed.

To publish topics, call MQTTController's extPublishTopic( topic, payload, opts ) method. The topic must be a valid topic string. The payload can be any type; if it is an object, array, or null, it will be sent as JSON3. Otherwise, it is simply converted to a string and sent. To send no payload at all, specify undefined as the argument (or omit it). The optional argument opts is an object that may specify the key qos with an integer value 0 through 2 (inclusive, default 0) and retain as a boolean (default: false).

    mqttcontroller.extPublishTopic( "mycontroller/status/OK" );
    mqttcontroller.extPublishTopic( "mycontroller/dimmer/level", 50 );
    mqttcontroller.extPublishTopic( "mycontroller/weather/current", { temp: 21, hum: 44 },
        { qos: 1, retain: true } );

This method returns a *Promise* that resolves when the topic has been sent, or rejects if an error occurs.

Updated: 2022-Dec-02


  1. Thank you to SmartHome Community user @Crille for contributing this template. 

  2. Thank you to SmartHome Community user @therealdb for the inspiration and basis of this template. 

  3. This implies, of course, that the object can be converted to JSON by JSON.stringify(), which means that it can contain only JavaScript primitive types (string, number, boolean, null), Date instances, and arrays and simple, prototype-less objects (i.e. not instances of defined classes, be they defined by you or JavaScript types like Map, Set, etc.) containing only these permitted types as elements/values.