ESP32 Classic MQTT Publisher & Home Assistant

[ClassicCrazy]

...
success on getting Android Classic Monitoring  App on MQTT 
( first had trouble getting mqtt working on my Fire android tablet - but after clearing , uninstalling, installing newest version , more stuff like restarting it I finally got it to work . Version says 3.0.6 . but I installed same APK on my galaxy 4 phone and it shows 3.1.0 version. )
Larry

Hmm, not sure why it's showing version 3.1.0, could your galaxy 4 already have had version 3.1.0, you would not be able to install 3.0.6 without first uninstalling 3.1.0.
The "MQTT" branch of ClassicMonitor is currently at version 3.0.6,
the new "master" branch (currently 3.1.4) is the merge of the latest updates of the app store branch "NoMQTT" with the MQTT branch as well as an experimental update to access my Tri-Star TS-45 using the modbus adapter.
Version 3.0.6 should be ok for testing ClassicMQTT


Graham

I probably had some other version of Classic Monitoring app on my phone already and didn't realize it- I don't use that phone too often as I have lineage OS on it and don't pay for phone service.

Larry

[Graham]

I updated the image on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt so that you can now define the startup parameters in the environment section of the docker-compose.yml file when using IOTStack to setup your docker containers. You no longer need the .env file or the command: parameters.
Updated the wiki accordingly...
https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack


Graham

[ClassicCrazy]

I updated the image on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt so that you can now define the startup parameters in the environment section of the docker-compose.yml file when using IOTStack to setup your docker containers. You no longer need the .env file or the command: parameters.
Updated the wiki accordingly...
https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack


Graham

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.

I had already deleted .env file , then I deleted the classic part of docker_compose.yml , I used portainer and removed previous classicmqtt image, rebooted pi , but it still won't build and I get the same error as above.
After breakfast I may try to do what the link in the error message says to do if I can figure out how to do that.

Larry

[Graham]

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=wt9m']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3
...
Larry

Looks like you have the container name in there twice

the second block should be called something different than the first and must have a unique container_name (see attached example)

you can delete your docker-compose.yml file in IOTStack, the menu.sh->Build Stack will create a new one with everything you selected.


Graham.

[ClassicCrazy]

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=wt9m']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3
...
Larry

Looks like you have the container name in there twice

the second block should be called something different than the first and must have a unique container_name (see attached example)

you can delete your docker-compose.yml file in IOTStack, the menu.sh->Build Stack will create a new one with everything you selected.


Graham.

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry

[Graham]

...

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry

thanks Larry
I updated the file


Graham

[ClassicCrazy]

...

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry

thanks Larry
I updated the file


Graham

Okay rebuilt worked .
I wasn't getting Classic2 publishing
Just found a typo I put in on the ip address
That took awhile for me to figure out !
Larry

[mcsarge]

Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt

[Graham]

Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt

Hey Matt, did you see my pull request https://github.com/ClassicDIY/ClassicMQTT/pull/25 that adds the environment variables...


Graham.

[mcsarge]

Well,

No I did not, stupid me. I put a bunch of work into the validator set them first and your method of setting them in the initialization looks far easier! I will get your changes and then push them down to me.

Matt

Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt

Hey Matt, did you see my pull request https://github.com/ClassicDIY/ClassicMQTT/pull/25 that adds the environment variables...


Graham.

[daklein]

Thanks McSarge & Graham for this great tool.  I have some limited data available from my inverters, but with classicmqtt I'm also able to get battery bank voltage, which makes it possible to roughly control some loads and make use of variable solar production, instead of losing it as the main battery is near full.

I redid some of my RPi with ClassicMQTT & IOTstack, after the sd card filled up and things broke.   I believe I had previously built the docker image & was using the .env & command line argument method to give ClassicMQTT some of the parameters.  It worked. 

I followed these instructions (Thanks!),  https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack
This time I think I am running the image from DockerHub, portainer shows me that it was built 1/31   
classicdiy/classicmqtt:latest   120.3 MB   2021-01-31 08:56:57

It is working, but,  the former .env file & command line arguments for publish rate,  now in the environment section of compose-override.yml seem to not have an effect.     It publishes just at the default 5 minute rate, I'm not sure what I'm missing?  I'm pretty sure it's seeing the compose-override.yml file I'm editing,  if I change the classic's ip addr, etc, that will break it or not.   

If i change the override file, and run the ./menu.sh script to build and start the IOTstack,  it does recreate the classicmqtt container, and the main docker-compose.yml gets text changes.

Code Select Expand


Start Stack:
docker-compose up -d --remove-orphans
WARNING: Some networks were defined but are not used by any service: vpn_nw, iotstack_nw_internal, nextcloud_internal
nodered is up-to-date
portainer-ce is up-to-date
grafana is up-to-date
influxdb is up-to-date
Recreating classic_mqtt ...
mosquitto is up-to-date
Recreating classic_mqtt ... done

Stack Started




Here's what's in the top of docker-compose.yml,  from the compose-override.yml file that was edited.

Code Select Expand


pi@hassbian:~/IOTstack $ more docker-compose.yml
version: '3.6'
services:
  classic_mqtt:
    container_name: classic_mqtt
    image: classicdiy/classicmqtt
    restart: unless-stopped
    environment:
    - LOGLEVEL=DEBUG
    - CLASSIC=192.168.16.10
    - CLASSIC_PORT=502
    - CLASSIC_NAME=MNC250CP
    - MQTT_HOST=mosquitto
    - MQTT_PORT=1883
    - MQTT_ROOT=ClassicMQTT
    - MQTT_USER=mqtt_user
    - MQTT_PASS=mqtt_user_pw
    - AWAKE_PUBLISH_RATE=3
    - SNOOZE_PUBLISH_RATE=5
    networks:
    - iotstack_nw

# uncomment the following block if you have a second classic, change the CLASSIC ip address and CLASSIC_NAME
#  classic_mqtt2:
#    container_name: classic_mqtt2
#    image: classicdiy/classicmqtt
#    restart: unless-stopped
#    environment:
#      - LOGLEVEL=DEBUG
#      - CLASSIC=192.168.86.137
#      - CLASSIC_PORT=502
#      - CLASSIC_NAME=MyShed
#      - MQTT_HOST=mosquitto
#      - MQTT_PORT=1883
#      - MQTT_ROOT=ClassicMQTT
#      - MQTT_USER=ClassicPublisher
#      - MQTT_PASS=ClassicPub123
#    networks:
#      - iotstack_nw

# Here is an example using an external MQTT broker
#  classic_mqtt:
#    container_name: classic_mqtt
#    image: classicdiy/classicmqtt
#    restart: unless-stopped
#    environment:
#      - LOGLEVEL=DEBUG
#      - CLASSIC=192.168.86.37
#      - CLASSIC_PORT=502
#      - CLASSIC_NAME=MyWorkshop
#      - MQTT_HOST=192.168.86.82
#      - MQTT_PORT=1883
#      - MQTT_ROOT=ClassicMQTT
#      - MQTT_USER=HASSUser
#      - MQTT_PASS=HASSpw
#    networks:
#      - iotstack_nw
  grafana:
    container_name: grafana
    image: grafana/grafana
    restart: unless-stopped
    user: "0"
    ports:
    - "3000:3000"
    environment:
    - GF_PATHS_DATA=/var/lib/grafana
    - GF_PATHS_LOGS=/var/log/grafana
    volumes:
    - ./volumes/grafana/data:/var/lib/grafana
    - ./volumes/grafana/log:/var/log/grafana
    networks:
    - iotstack_nw

  python:
    container_name: python
    build: ./services/python/.
    restart: unless-stopped
    network_mode: host
    volumes:
    - ./volumes/python/app:/usr/src/app
  portainer-ce:

.  etcetera
.      etcetera
.           etcetera


[Graham]

...
It is working, but,  the former .env file & command line arguments for publish rate,  now in the environment section of compose-override.yml seem to not have an effect.     It publishes just at the default 5 minute rate, I'm not sure what I'm missing?  I'm pretty sure it's seeing the compose-override.yml file I'm editing,  if I change the classic's ip addr, etc, that will break it or not.   

I pushed a new image to dockerHub, try re running "Build Stack" and "Start Stack" in IOTStack/menu.sh
I also had to set the SNOOZE_PUBLISH_RATE to 300

check the logs in portainer for classic_mqtt to verify the startup arguments

let me know if this works...
Graham.

[daklein]

I pushed a new image to dockerHub, try re running "Build Stack" and "Start Stack" in IOTStack/menu.sh
I also had to set the SNOOZE_PUBLISH_RATE to 300

check the logs in portainer for classic_mqtt to verify the startup arguments

let me know if this works...
Graham.

Graham,   Thanks!    It does work.  It got the new image from dockerhub, and it's publishing at the faster rate I was looking for.

I set both the wake & snooze interval to 3 sec, and it seems to be populating into influxdb at that rate now.

The log shows it going in & out of snooze mode, opening and closing the connection to the Classic.  Is that ok, or likely to cause some issue?   Is snooze meant to be nighttime, no solar production, for time > WAKE_DURATION.
   
(An issue beyond the need to downsample longer term data in database, which I need to figure out eventually.  But I want the relatively faster rate to use for control, using battery voltage vs. target voltage.)

Code Select Expand


2021-05-11 21:31:07,934:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:07,935:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:07,938:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings
2021-05-11 21:31:07,940:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/info
2021-05-11 21:31:13,920:DEBUG:classic_mqtt:Opening the modbus Connection
2021-05-11 21:31:13,940:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:13,941:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:13,945:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings
2021-05-11 21:31:13,949:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/info
2021-05-11 21:31:19,923:DEBUG:classic_mqtt:Opening the modbus Connection
2021-05-11 21:31:19,942:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:19,944:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:19,948:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings


[Graham]

...
The log shows it going in & out of snooze mode, opening and closing the connection to the Classic.  Is that ok, or likely to cause some issue?   Is snooze meant to be nighttime, no solar production, for time > WAKE_DURATION.
   

SNOOZE_PUBLISH_RATE should be around 300 seconds since, in this mode, it will disconnect from modbus after every read to allow the local app to get in and access the classic.
If you want it to stay awake you can send a MQTT cmnd with a json payload containing "stayAwake"

ClassicMQTT/MNC250CP/cmnd {"stayAwake"}

[ljane]

Reviving this topic after a long time because I'm not sure my question warrants a completely new topic.

Am I right in understanding that the Monitor app won't work with MQTT unless you also have a RPi or ESP32 publishing the data?

I was initially under the impression that the newer Android app allowed us to bypass adding the RPi, but I think I finally figured out that was incorrect. Before I completely throw in the towel (just don't have the bandwidth to learn yet another piece of technology), I wanted to make sure I'm not missing something.

The good news is that I've read the readmes and wikis enough times now that I think I understand more words than I don't understand! (I'm an accountant, not an engineer.)

Thanks!