Record MQTT topics to a MikroTik router's SSD

Steps#

1. Create an envlist that holds only the MQTT login. Write the two lines to a local file, upload it, import it, then delete it on the router and locally:

   /container/envs/add list=mqttrec key=OBSERVER_MQTT_USER value="<MQTT_USER>"
   /container/envs/add list=mqttrec key=OBSERVER_MQTT_PASSWORD value="<MQTT_PASSWORD>"
   

   chmod 600 mqttrec-env.rsc
   scp mqttrec-env.rsc "<ROUTER>:/<DISK>/"
   ssh <ROUTER> '/import file-name=<DISK>/mqttrec-env.rsc'
   ssh <ROUTER> '/file/remove [find name="<DISK>/mqttrec-env.rsc"]'
   rm mqttrec-env.rsc
   

   Check the two keys without printing their values: ssh <ROUTER> ':put [:len [/container/envs/find list=mqttrec]]'. Expected: 2 [UNVERIFIED: not run in this form]. The key names are the ones run.sh reads; if you rename them, rename them there too.

   GotchaDon't reuse another container's envlist

   A container gets every variable in each list it names. My first deploy reused the envlist of another container on the router to avoid a new secret; that list also held alert-bot tokens and other passwords, so a stock Docker Hub image could read all of them. I switched it to this two-key list the same day.

2. Write run.sh. This is the script as deployed, without its header comments. Change the broker address and the topics (-t) to yours:

   mkdir -p /rec/log
   # sh as PID 1 ignores SIGTERM: forward it, or a container stop waits for the kill timeout
   trap 'kill $pid 2>/dev/null; exit 0' TERM INT
   while :; do
     f=/rec/log/mqtt-$(date -u +%Y-%m-%d).log
     mosquitto_sub -h 172.17.0.3 -u "$OBSERVER_MQTT_USER" -P "$OBSERVER_MQTT_PASSWORD" -k 60 -W 3600 \
       -F '%I %t %p' -t 'ups/+/state' -t 'ups/+/bridge' -t 'ups/ac180/info' -t 'ups/ac180/log' >> "$f" &
     pid=$!
     wait $pid
     sleep 1
   done
   

   What each part does:

   • -F '%I %t %p' prints an ISO-8601 timestamp in the container's time zone (with its offset), the topic and the raw payload on one line.
   • -W 3600 makes mosquitto_sub disconnect an hour after it connects. Only then does the loop compute the file name again, so the switch to the next day's file happens at the first reconnect after 00:00 UTC, up to an hour late: lines from the last hour before that reconnect still go into the previous day's file. Cost: about one second without a subscriber per hour, and retained messages are written again after every reconnect.
   • -k 60 is the keepalive; 60 s is also the default.
   • mosquitto_sub runs in the background with wait, and the trap kills it on TERM. Without that, sh is PID 1 and ignores SIGTERM: in Docker a stop then waits 10 s and kills the container, exit code 137 (Failure modes, row 3).
   • Errors from mosquitto_sub go to stderr, which the router's log shows when the container has logging=yes.

3. Upload run.sh to the directory the container will mount.

   scp run.sh "<ROUTER>:/<DISK>/mqtt-recorder/run.sh"
   

   Expected: /<DISK>/mqtt-recorder/run.sh is listed by ssh <ROUTER> '/file/print where name~"mqtt-recorder"'. [UNVERIFIED: whether scp creates a missing mqtt-recorder directory on RouterOS 7.24] If the file is not listed, create the directory on the disk first (WinBox or WebFig, Files) and upload again.

4. Write the setup script mqtt-recorder-setup.rsc. This is my script file as it stands after the envlist switch, minus two comment lines about the envlist; set the :local values to yours:

   # MQTT value recorder: mosquitto_sub on the router writes every ups/* message to the SSD
   # (/usb1-part1/mqtt-recorder/log/mqtt-YYYY-MM-DD.log). The router stays up through outages.
   # Router-side state: veth 172.17.0.5, bridge port, mount, container. No NAT (outbound to the broker only).
   :local disk "usb1-part1"
   :local ctBridge "bridge-containers"
   :local ctAddr "172.17.0.5"
   :local ctGw "172.17.0.1"
   :local image "library/eclipse-mosquitto:2.1.2-alpine"
   /interface/veth/add name=veth-mqttrec address=($ctAddr . "/24") gateway=$ctGw comment=mqtt-recorder
   /interface/bridge/port/add bridge=$ctBridge interface=veth-mqttrec comment=mqtt-recorder
   /container/mounts/add list=mqtt-recorder src=("/" . $disk . "/mqtt-recorder") dst=/rec comment=mqtt-recorder
   /container/add name=mqtt-recorder remote-image=$image interface=veth-mqttrec \
       root-dir=("/" . $disk . "/mqtt-recorder/root") mountlists=mqtt-recorder envlists=mqttrec \
       cmd="/bin/sh /rec/run.sh" \
       start-on-boot=yes restart-policy=always restart-interval=10s logging=yes memory-high=32MiB comment=mqtt-recorder
   

   The image's root goes under the same directory as run.sh and the logs; only /<DISK>/mqtt-recorder is mounted into the container, as /rec. Keep restart-policy=always and start-on-boot=yes so the recorder comes back after a crash or a router reboot. No NAT rule is needed: the container only connects out to the broker.

5. Upload the script, dry-run it, then import it.

   scp mqtt-recorder-setup.rsc "<ROUTER>:/<DISK>/"
   ssh <ROUTER> '/import file-name=<DISK>/mqtt-recorder-setup.rsc verbose=yes dry-run'
   

   Expected: No syntax errors found in the import file. Without verbose=yes RouterOS 7.24 refuses the dry run (Failure modes, row 1). Then import for real, without verbose=yes dry-run:

   ssh <ROUTER> '/import file-name=<DISK>/mqtt-recorder-setup.rsc'
   

   The add starts the image pull and extraction into root-dir.

6. Start the container once the image is extracted. MikroTik's page: "the Container itself will not be started after it has been added, you need to start it manually for the first time after it has been downloaded/extracted."

   ssh <ROUTER> '/container/start [find name=mqtt-recorder]'
   ssh <ROUTER> ':put [/container/get [find name=mqtt-recorder] running]'
   

   Expected: true [UNVERIFIED: not run in this form]. If start fails while the pull is still running, wait and repeat; Update a RouterOS 7 container from a new image tar has a retry loop for this.

7. Read the container's log once.

   ssh <ROUTER> '/log/print where topics~"container" and message~"mqtt-recorder"'
   

   One Error: Bad file descriptor right after the start is harmless: the first connect ran before the veth was up, and the loop connected one second later (Failure modes, row 2).