Skip to content

MQTTController

MQTTController is the Reactor interface to the world of MQTT. It requires a MQTT broker running somewhere on the network (MQTTController is not itself a broker, it's a client). The MQTT broker must be fully MQTT 3.1.1 compliant, or higher. Please note that as of this writing (see date at bottom of the page), the openLuup MQTT controller is not fully 3.1.1 compliant, so while it may be suitable for testing and learning, it should not be deployed for "production" use.

There are two functions that MQTTController performs at the highest level of perspective:

  1. It maps devices that send and receive MQTT events into Reactor entities. For example, a Shelly1 relay device or an ESP8266 flashed with Tasmota can connect to an MQTT broker and exchange messages. Reactor connected to the same broker will "hear" status updates published by connected devices, and can update attributes on a Reactor entity accordingly (e.g. set the temperature_sensor.value attribute to the value of the temperature published), and actions performed on the Reactor entity can be turned into command messages to the device.
  2. It can "echo" any entity in Reactor to the MQTT universe, effectively acting as a gateway so that all entities known to Reactor are published. Commands can be issued to Reactor using MQTT to run actions on the entities, and attribute (state) changes on the entities in Reactor are published to the MQTT broker.

Installation

MQTTController is not included in base Reactor distributions. It must be installed separately by downloading the package from the extras subdirectory of the Reactor download server.

Once the package file (a ZIP or Gzip'd tar archive) is downloaded:

  1. Create, if it does not already exist, a directory called ext in your Reactor install directory (so it should be at the same level as config, storage, etc.).

    cd /path/to/reactor
    mkdir ext
    

    If you are running Reactor in a docker container, the ext directory should be created in the data directory, where your config and storage directories live as indicated above.

  2. Change directory into the new ext directory: cd ext

  3. Unpack the archive (use whichever command is correct for the type of archive downloaded):

    tar xzvf /path/to/MQTTController-XXXXX.tar.gz    # or...
    unzip /path/to/MQTTController-XXXXX.zip
    
  4. You should now have a directory called MQTTController within ext. Change directory into it:

    cd MQTTController
    
  5. Run the install script:

    ./install.sh
    

From here, proceed to Basic Configuration below.

Basic Configuration

In order to use MQTTController at all, you have to add an entry for it to the controllers section of your reactor.yaml file.

controllers:
  # Your existing controllers will be below the above line.
  # Add the following after the last "- id" line in this
  # section.
  - id: mqtt
    name: MQTT
    enabled: true
    implementation: MQTTController
    config:
      # Replace IP with that of your MQTT broker below
      source: "mqtt://127.0.0.1:1883/"

Restart Reactor to make the changes take effect. After that, you should be able to refresh the UI, go the Entities list, clear any existing filters, and choose "MQTT" from the controllers filter selector. That should then show you two entities: the MQTT controller system entity, and its default group entity. If you don't see this, check the log for errors.

If you need to authenticate with your MQTT broker, you can supply username and password fields at the same level as source.

Configuring Devices

Devices are configured within an entities subsection of the config section for MQTTContoller.

MQTTController uses a data-driven model to map published messages to attributes on entities, and entity actions to published commands. There are two ways to map a device:

  1. If an existing preconfigured template will work for your device, you provide the topic that identifies your device, and tell MQTTController what template to apply to it.
  2. If there is no existing template that works for your device, you create on by telling MQTTController what Reactor capabilities the device has, and how to map published messages with state/telemetry updates into the attributes of those capabilities. You will also tell Reactor how to take actions in those capabilities and map them to topics to be published to make the device do what you need it to do.

The preconfigured templates are located in mqtt_devices.yaml, in the templates section. Do not modify this file. This is a system file. There is a separate mechanism for you to define your own templates without modifying the system file.

The currently known templates are:

  • tasmota_generic_relay - Maps payload data and actions to the entity (capabilities switch_power and toggle); use optional unit to specify relay number if the device supports/reports more than one (default: blank/no unit).
  • tasmota_detached_button - Maps payload data for a detached switch input to the entity (capability button); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit).
  • tasmota_detached_switch_mode0 - Maps payload data for a detached switch in mode 0 (toggle/momentary; capability button); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit).
  • tasmota_detached_switch_mode1 - Maps payload data for a detached switch in mode 0 (on/off; capability binary_sensor); use optional unit to specify switch number if the device supports/reports more than one (default: blank/no unit).
  • tasmota_sensor_temperature_humidity - Maps temperature and humidity values from payload into a single entity (capabilities temperature_sensor and humidity_sensor); use source to identify sensor subunit.
  • tasmota_sensor_temperature - Maps temperature (only) from payload into the entity (capability temperature_sensor); use source to identify sensor subunit.
  • tasmota_sensor_humidity - Maps humidity level (only) from payload into the entity (capability humidity_sensor); use source to identify sensor subunit.
  • tasmota_sensor_battery - Maps payload battery data to the entity; requires source to identify the sensor subunit.
  • tasmota_sensor_co2 - Maps payload carbon dioxide level data to the entity; requires source to identify the sensor subunit.
  • tasmota_sensor_dewpoint - Maps payload dewpoint to the entity; requires source to identify the sensor subunit.
  • tasmota_sensor_rssi - Maps payload RSSI to the entity; requires source to identify the sensor subunit.
  • shelly_relay - Creates an entity that maps one relay channel of a Shelly node to an entity.
  • shelly_input - Creates an entity that that maps one input channel of a Shelly node to an entity.
  • shelly_temperature - Creates an entity to contain the temperature sensor data from a Shelly device.
  • shelly_humidity - Creates an entity to contain the humidity sensor data from a Shelly device.
  • shelly_motion - Creates an entity to reflect the motion/no motion state of a Shelly motion sensor.
  • shelly_smoke - Creates an entity to reflect the state of a Shelly smoke detector.
  • shelly_flood - Creates an entity to reflect the dry/wet state of a Shelly flood sensor.
  • shelly_door_window - Creates an entity to reflect the opened/closed state of a Shelly flood sensor.
  • shelly_battery - Creates an entity to contain the battery level of a battery-operated Shelly device (e.g. a Shelly HT).
  • shelly_power - Creates an entity with a Shelly device's power (watts) measurement.
  • shelly_tilt - Creates an entity to contain the measurement of a Shelly device's tilt sensor.
  • shelly_vibration - Creates an entity to contain the measurement of a Shelly device's tilt sensor.

Easy Device Configuration - Using Existing Template

As an example, let's say we have a Shelly1 configured with the topic shelly1_43DFD2 that is configured to operate as a single relay to operate a stairway light. To create an entity for this device, we can use the tasmota_generic_relay template. The template supports multiple relays, but since only one relay is configured in Tasmota, the published topics from the node will have a "unit number" (identifying which relay if more than one), so we'll specify no unit number by giving it an empty string:

...other stuff
    config: "mqtt://127.0.0.1:1883/"
    entities:
      shelly1_stairway:
        name: "Stairway Light"
        topic: shelly1_43DFD2
        uses_template: shelly_relay
        channel: 0

After applying this configuration, we restart Reactor, and then refresh the UI. We should then be able to find the Stairway Light entity in the Entities list. You should be able to use the power_switch.on and .off actions to turn it on and off, and observe that the state correctly tracks that of the device, even when operated from the Shelly mobile app or any other means.

Detailed Device Configuration

If an existing template doesn't work, you can configure the device directly. Later, if you wish, you can turn your configuration into a reusable template, so if you have many similar devices, the configuration can be shared between then.

To get MQTTController to create and maintain an entity for your MQTT-enabled device, you need to create a mapping configuration that tells MQTTController:

  • What type of entity to create;
  • What capabilities it will have;
  • How to turn state/status topics received for the node into attribute values on the entity;
  • How to turn action requests for the entity into published MQTT messages.

To illustrate how this is done, let's implement a configuration for a Tasmota in generic/relay configuration (there's a template for this, but we're doing it explicitly here to show how that's done). Our Tasmota device is configured for a single relay, basically a switch, so we're going to map it to a Switch type entity with the two capabilities that most switches have in Reactor: power_switch and toggle.

We'll begin with our basic entity configuration. First, we need to assign a unique ID for the entity. To make it easy, we're going to use the Tasmota node's ID:

  - id: mqtt
    config:
      source: "mqtt://127.0.0.1:1883/"
      entities:
        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"

Now we can get into configuring what the entity can do:

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
           capabilities:
             - power_switch
             - toggle
           primary_attribute: power_switch.state

OK, the above additions tell MQTTController that we're going to create a Switch type entity, and it will have capabilities power_switch and toggle. Its primary attribute will be power_switch.state.

Info

Don't worry if you don't recognize the Switch entity type, or know what you should put there. This value is used by the Dashboard to guess a default configuration for tiles, but since the Dashboard isn't really the focus of work right now, you can ignore this value. In fact, you can leave it out altogether; it won't hurt anything.

The power_switch capability is the only one of the two being assigned that has any attributes, and its only attribute is that state attribute being assigned as the primary. We need to get the value out of status update topics we receive and into that attribute. Tasmota sends topics of the form stat/nodeID/POWER for the status of a single relay device. When the relay is turned on, the payload is the string on, and when it's turned off, the payload is off. We'll create an events section, and put a handler in for that topic.

         ...
         primary_attribute: power_switch.state
         events:
           "stat/%topic%/POWER":
             "power_switch.state":
               expr: 'bool(payload)'

The first thing to notice is the %topic% string in the event topic. This is a substitution: MQTTController will put the string in the configuration value topic for this entity into that string. We could also have just hard coded it (that is, use stat/tasmota_FFFFFF/POWER), but if we want to make a template out of this later, it's better to use the substitution.

So having told MQTTController what topic this entity is interested in, we then tell it what attributes that topic can effect. This topic only affects one attribute, power_switch.state, and we use an expression to convert the payload string (on or off as stated above) to the boolean type that is required for power_switch.state. The expression here is trivially easy because the function bool() recognizes and converts the string on, yes and true to true, and off, no and false to false, and is not case-sensitive.

If that topic could affect more attributes, we would list them using the same structure as that described for power_switch.state.

Likewise, if there were other topics that could affect attributes (including other topics that would affect power_switch.state), we could just list them in the events section and structure them as we have done so far. MQTTController will make sure every topic gets processed by every entity that is interested in it, and every attribute that can be affected by it.

Now let's move on to actions. In the actions section, we list each capability that has actions, and within each of those, we list each action and its implementation. Let's start with power_switch.on and power_switch.off. These simple actions take no parameters, so their implementation is very simple:

         events:
            ...
         actions:
           power_switch:
             "on":
               topic: "cmnd/%topic%/Power"
               payload: "on"
             "off":
               topic: "cmnd/%topic%/Power"
               payload: "off"

Under each action, the required value topic sets the topic to be published to implement the action. If the topic has a payload, then the payload value can be added. The payload shown here for both actions is just a string, because the payload for these commands is a string, and in this case, it's a fixed string (it is always on for the on action and off for the off action. The power_switch.set action makes things a little more complex, because it takes a boolean parameter named state to determine what to do with the light/switch/relay. Here's that implementation:

         actions:
           power_switch:
             ...
              set:
                topic: "cmnd/%topic%/Power"
                payload:
                  expr: "parameters.state ? 'on' : 'off'"
                  type: raw

In this implementation, the topic is the same, but the payload is more dynamic. The paylaod value is an object with two keys. The expr key/value is an expression that figures out whether to return the string on or off based on the value of the state parameter. The type key/value of raw tells MQTTController that the string returned by the expression should not receive any translation. So the payload thus becomes the string on or off sent with this topic to implement this action.

Some topics have more complex payloads. It's not usual for a topic to have a JSON payload, for example. In this case, the expr value may return an object, and if the type is set to json, MQTTController will JSONify the expression result to a string that is then sent as the payload.

If your action wants to set the MQTT qos level or retain flag, just provide those using keys of the same name. The value of qos should be an integer, and retain should be boolean.

Since we've done this in chunks, here's the whole configuration all together:

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
          capabilities:
            - power_switch
            - toggle
          primary_attribute: power_switch.state
          events:
            "state/%topic%/POWER":
              "power_switch.state":
                expr: 'bool(payload)'
          actions:
            power_switch:
              "on":
                topic: "cmnd/%topic%/Power"
                payload: "on"
              "off":
                topic: "cmnd/%topic%/Power"
                payload: "off"
              set:
                topic: "cmnd/%topic%/Power"
                payload:
                  expr: "parameters.state ? 'on' : 'off'"
                  type: raw

Notice anything missing? We haven't defined the toggle.toggle action. The toggle capability has no attributes and just that one action, but we haven't defined what that action does. As it turns out, MQTTController can supply a default implementation for toggle.toggle, so we don't need to define it.

Additional Entity Configuration

The query key can be used to provide a topic that causes the device to report its current state. A payload is not allowed, only a topic (string). When this optional key is provided, the x_mqtt_device.poll action is enabled and will cause the query topic to be sent to prompt a device update.

The init key may be used to send initialization strings to a device. These strings will be sent every time Reactor connects to the MQTT broker. The init value may be:

  1. A string, in which case the topic (with no payload) is the only topic sent at initialization;
  2. An array, containing any number and mix of:
    • string, which are topics to be sent (with no payload);
    • object, containing the required key topic and the optional keys payload, qos and retain.

Here's an example of how these are formatted in the configuration (when needed):

        "tasmota_FFFFFF_relay":
          name: "Tasmota Relay"
          topic: "tasmota_FFFFFF"
          type: Switch
          capabilities:
            - power_switch
            - toggle
          primary_attribute: power_switch.state
          query: "cmnd/%topic%/Power"
          init:
            - "cmnd/%topic%/someAction"
            - topic: "cmnd/%topic%/anotherAction"
              payload:
                level: 50
                rate: 3
              type: json
              qos: 0
              retain: false

Converting Your Configuration into a Template

In the previous section, we built the device configuration directly into our per-device configuration. If we had several such devices, though, it would be verbose and more difficult to maintain all of them with each device having a complete copy of all those lines. We can simplify our configuration by converting our device configuration into a template.

To begin, create a file in your config subdirectory called local_mqtt_devices.yaml. Into that, we'll create a templates section, and within that, we'll assign an identifier (name) to our template. We'll call it my_tasmota_relay:

templates:
   # Optional description of what your template does (this is a comment)
   my_tasmota_relay:

Now, we can basically copy the device configuration, leaving behind the id, name and topic. That is, we want to move the type, capabilities, query, events, and actions sections to the template. It should then look like this (you'll need to adjust indentation, because the levels change):

templates:
  # Optional description of what your template does (this is a comment)
  my_tasmota_relay:
    type: Switch
    capabilities:
      - power_switch
      - toggle
    primary_attribute: power_switch.state
    events:
      "state/%topic%/POWER":
        "power_switch.state":
          expr: 'bool(payload)'
    actions:
      power_switch:
        "on":
          topic: "cmnd/%topic%/Power"
          payload: "on"
        "off":
          topic: "cmnd/%topic%/Power"
          payload: "off"
        set:
          topic: "cmnd/%topic%/Power"
          payload:
            expr: "parameters.state ? 'on' : 'off'"
            type: raw

Then, in your device configuration, you can refer to your template with the uses_template key. Here's our revised device configuration, with an additional entity of the same type added for another Tasmota relay node, using the template for both:

  - id: mqtt
    ...
    config:
      source: "mqtt://127.0.0.1:1883/"
      entities:
        "tasmota_FFFFFFF_relay":
          name: "Tasmota Relay 1"
          topic: "tasmota_FFFFFF"
          uses_template: my_tasmota_relay
        "tasmota_BEEFEE_relay":
          name: "Tasmota Relay 2"
          topic: "tasmota_BEEFEE"
          uses_template: my_tasmota_relay

Tip

All topic strings will be put through substitution, and the keywords in the substititions are values from the device configuration (like topic). If you have devices with multiple relays, add keywords like channel or unit, mirror whatever the device manufacturer tends to use, so that you effectively parameterize your template and let it work the same for multiple instances of a device's components. To help clarify this, look at the implementation of the "real" shelly_relay template in MQTTController's mqtt_devices.yaml file.

Echo Capability

MQTTController can, if configured, publish messages for every state change of a set of entities you choose (or all entities). In this way, all entities known to Reactor may become MQTT-enabled, regardless of their source. MQTTController also knows how to receive topics to perform actions on these entities.

Info

I'm not sure what to call this function, so for now I'm just calling it "echo" capability. Gateway...? Relay...? Great and Powerful Oz? Any ideas?

Enable echoing by defining an echo section in your MQTTController configuration:

  - id: mqtt
    ...
    entities:
      ...
    echo:
      include_entities: []
      exclude_entities: []
      include_capabilities: []
      exclude_capabilities: []
      primary_attributes: true

The include_entities key is an array (define using either YAML format; the other form is shown in the sample below) containing canonical ID patterns to match. The patterns are defined as follows:

  • Literal canonical IDs, like hubitat>100, to refer to a single entity;
  • Wildcard to match all entities on a particular controller: hubitat>* matches all entities from the controller with ID hubitat;
  • Regular expressions: /pattern-regexp/

For example (using alternate array syntax for improved readability):

    echo:
      include_entities:
        - "vera>*"
        - "hass>*"

This would match all entities from the controllers with IDs vera and hass. If included_entities is not specified (i.e. commented out or not present in the config), then all entities from all controllers are included.

After inclusion, exclude_entities is processed in the same manner as include_entities, except that any entity that matches will be discarded.

From the final set of entities to echo, the include_capabilities and exclude_capabilies will determine what capabilities are sent as topics. For each capability on an entity to be sent, the topic will be reactor/<mqtt-id>/<ctrl-id>/<entity-id>/<capability-name>, and the payload is an object containing the attribute key/value pairs for that capability on that entity. The mqtt-id is the ID of the MQTTController, the entity-id is the ID of the entity that is the subject of the message, and the ctrl-id is the controller ID to which that entity belongs.

reactor/mqtt/vera/device_10/power_switch { "state": true }

The setting of primary_attributes in the echo section determines whether a generic value topic is sent for the primary attribute of the entity. If this is true you will see topics of the form reactor/<mqtt-id>/<ctrl-id>/<entity-id>/value with a payload string containing the primary attribute value. For a switch the primary attribute of which is power_switch.state, one might expect:

reactor/mqtt/hass/switch_living_room/value true

Commanding Reactor Entities with MQTT

The last function of the echo capability is to relay actions from MQTT topics to Reactor entities. To run an action on an entity, publish a topic to Reactor in this form: reactor/<mqtt-id>/<ctrl-id>/<entity-id>/perform/<action-capability>/<action-name>. If the action takes parameters, encode the parameter key/value pairs as a JSON payload. Here's an action to turn on our fictional living room light:

reactor/mqtt/hass/switch_living_room/perform/power_switch/on (no payload)
reactor/mqtt/hass/switch_living_room/perform/power_switch/set { "state": true }

Both of the above topics accomplish the same task; one action takes parameters, the other does not. Here's a more extreme example, setting color on an RGB bulb:

reactor/mqtt/vera/device_188/perform/rgb_color/set_rgb { "red": 255, "green": 128, blue: 16 }

Updated: 2021-Nov-26