ESP32 Classic MQTT Publisher & Home Assistant

[ClassicCrazy]

Graham/Larry/Matt,   Thanks for setting up this Classic to MQTT tool, and for the great packaged setup.  It works!

Installed on a Pi with crossover cable to the Classic, and the Pi is set up as bridge/router from home wifi.  The Pi is set up using IOTstack with MQTT to NodeRed to influxdb to grafana.  The Pi collects data from 2 Enphase Envoys, a bridge to the utility main meter, a pair of SMA inverters, and now the Classic!  Woohoo!

I've been trying to figure out the SMA CAN messages enough to be useful for controlling (haven't found battery voltage & current signals correctly yet).  It's been on my list to connect up to the Classic, but it dawned on me that it has battery voltage.   It didn't take long to plug in and get it working.   

Glad you got it all working . Could you post a screen shot of the results you get on grafana ? I have been wanting to do that so any details you can share would be appreciated.  Since you are taking the data to Node Red could you share your flow and maybe that will show enough of how to make that setup work?   I also never thought about setting up the pi as a router .  Thanks

Larry

[ForeverHomestead]

Looks like it is working with one classic:

classic_mqtt_1  | 2021-01-24 08:18:46,794:DEBUG:classic_mqtt:Got data from Classic at 192.168.2.238:502
classic_mqtt_1  | 2021-01-24 08:18:46,804:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/readings
classic_mqtt_1  | 2021-01-24 08:18:46,807:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/info
classic_mqtt_1  | 2021-01-24 08:23:51,797:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-24 08:23:51,912:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
classic_mqtt_1  | 2021-01-24 08:23:51,913:DEBUG:classic_mqtt:Got data from Classic at 192.168.2.238:502
classic_mqtt_1  | 2021-01-24 08:23:51,916:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/readings
classic_mqtt_1  | 2021-01-24 08:23:51,917:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/info
classic_mqtt_1  | 2021-01-24 08:28:56,874:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-24 08:28:57,133:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode

I will work on getting the other items installed on the pi

[Graham]

You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82

[ForeverHomestead]

You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82

That is perfect thanks for the info.  I will work on that once I know that the first is working.  I have yet to get the info that it is publishing yet.  The app is not displaying any info.

I just got Portainer installed.  Still working on fuguring out how to view what was published.

[ForeverHomestead]

Ok, I have it working with the first of 3 Classics. 

How do I delete a device from the app?  I entered the wrong name the first time.

[Graham]

Ok, I have it working with the first of 3 Classics. 

How do I delete a device from the app?  I entered the wrong name the first time.

just slide it to the right

[ForeverHomestead]

just slide it to the right

Well wasnt that simple.  lol

[ForeverHomestead]

You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82

So if I run the file l;ike this. (Looks like example 2 in the how to)  Do I still need to make the .env file and run the build and up like in the 3rd example?

[ClassicCrazy]

If you do the two Docker instances like Graham said then make your .env file like I had in my screenshot . Just make sure that what you name your Classics matches what you have in the .env  ( case sensitive )

Larry

[Graham]

...

So if I run the file l;ike this. (Looks like example 2 in the how to)  Do I still need to make the .env file and run the build and up like in the 3rd example?

no, i"m passing in the arguments as parameters as in "2 Using docker" from the wiki so you don't need the env file

I'm assuming you already have the docker image built for ClassicMQTT , if not just run

docker build -t classic_mqtt .

also assuming you have the mosquito broker image running, use portainer to see what you have running...

if you want to use the "3. Using docker-compose" method then you would need to change the .yml like Larry has it in his screenshot

[ForeverHomestead]

That is perfect.  So the last remaining step for me is a way to get this data in to my home automation server.  I use a program called Homeseer.  To get the information from my Mate 3 I was able to write a script to parse the Json output of the mate.  What are my options for getting the information sent to Mosqitto?  is there a viewable output via web or somehng similar?

[Graham]

That is perfect.  So the last remaining step for me is a way to get this data in to my home automation server.  I use a program called Homeseer.  To get the information from my Mate 3 I was able to write a script to parse the Json output of the mate.  What are my options for getting the information sent to Mosqitto?  is there a viewable output via web or somehng similar?

You could use NodeRed to subscribe and publish to your mosquito broker

https://cookbook.nodered.org/#mqtt

I see node red has a homeseer plugin

https://flows.nodered.org/node/node-red-contrib-homeseer

if you just want to view data in the broker you can use http://mqtt-explorer.com/ or https://mqttfx.jensd.de/

[ForeverHomestead]

everything working on first Classic.  Tried adding the other two based on the image shown above and I get an error.  Any ideas what is wrong>

Code Select Expand

    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue.
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #

    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${M$
classic_mqtt2:
   image: classic_mqtt
   restart: always
   build: .
   environment:
     - LOGLEVEL=DEBUG
   depends_on:
     - mosquitto
   networks:
     - localnet
   command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass $
classic_mqtt3:
   image: classic_mqtt
   restart: always
   build: .
   environment:
     - LOGLEVEL=DEBUG
   depends_on:
     - mosquitto
   networks:
     - localnet
   command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass $
networks:
  localnet:


error is:

ERROR: yaml.parser.ParserError: while parsing a block mapping
  in "./classic_mqtt_compose.yml", line 1, column 1
expected <block end>, but found '<block mapping start>'
  in "./classic_mqtt_compose.yml", line 50, column 2

[ForeverHomestead]

So I had some spacing issues in the above.  I ran it through a validator and got that fixed, now the error has changed to:

ERROR: The Compose file './classic_mqtt_compose.yml' is invalid because:
'classic_mqtt2', 'classic_mqtt3' do not match any of the regexes: '^x-'

You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/


Any ideas?

[daklein]

Graham/Larry/Matt,   Thanks for setting up this Classic to MQTT tool, and for the great packaged setup.  It works!

Installed on a Pi with crossover cable to the Classic, and the Pi is set up as bridge/router from home wifi.  The Pi is set up using IOTstack with MQTT to NodeRed to influxdb to grafana.  The Pi collects data from 2 Enphase Envoys, a bridge to the utility main meter, a pair of SMA inverters, and now the Classic!  Woohoo!

I've been trying to figure out the SMA CAN messages enough to be useful for controlling (haven't found battery voltage & current signals correctly yet).  It's been on my list to connect up to the Classic, but it dawned on me that it has battery voltage.   It didn't take long to plug in and get it working.   

Glad you got it all working . Could you post a screen shot of the results you get on grafana ? I have been wanting to do that so any details you can share would be appreciated.  Since you are taking the data to Node Red could you share your flow and maybe that will show enough of how to make that setup work?   I also never thought about setting up the pi as a router .  Thanks

Larry

Tonight I got it working in the IOTstack system, running from docker-compose, with some help from the IOTstack discord.  Next it may get included as a selection in IOTstack.   

For now, I added just battery voltage measured from the Midnite, to the plots that mostly have data from the SMA inverters with fewer signals than I want properly scaled. Battery voltage will be very helpful, plus I'll add in the Midnite solar production to the plots.   

Here's a plot attached, Great fun!!     
      Little red line in upper plot, MN_BatV
      Upper plot timescale is zoomed, lower two plots are wider timescale
      The spikes are the SMA inverters going in & out of 'Silent' mode, charging for a short time w/ battery pretty full alreadt, then letting it sit with inverters off, while connected to the grid.

Here's the nodered flow I used to go from the mqtt message to influxdb:   
[{"id":"a11c0bc7.6343a8","type":"mqtt in","z":"502d1323.9f228c","name":"stat/readings","topic":"ClassicMQTT/MNC250CP/stat/readings","qos":"0","datatype":"json","broker":"724ca09d.3fbec","x":150,"y":140,"wires":[["6d87693e.9e2c98","835d8d89.09859"]]},{"id":"6d87693e.9e2c98","type":"debug","z":"502d1323.9f228c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":350,"y":180,"wires":[]},{"id":"944e24d5.0151c8","type":"influxdb out","z":"502d1323.9f228c","influxdb":"98b7f44b.9b4d48","name":"influx","measurement":"MN_BatV","precision":"","retentionPolicy":"","database":"Energy","precisionV18FluxV20":"s","retentionPolicyV18Flux":"","org":"organisation","bucket":"bucket","x":670,"y":140,"wires":[]},{"id":"835d8d89.09859","type":"change","z":"502d1323.9f228c","name":"MN_BatV","rules":[{"t":"set","p":"payload","pt":"msg","to":"{   \"MN_BatV\":payload.BatVoltage}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":140,"wires":[["d803ad69.c1a8f","944e24d5.0151c8"]]},{"id":"d803ad69.c1a8f","type":"debug","z":"502d1323.9f228c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":630,"y":180,"wires":[]},{"id":"724ca09d.3fbec","type":"mqtt-broker","name":"","broker":"192.168.15.15","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"98b7f44b.9b4d48","type":"influxdb","hostname":"127.0.0.1","port":"8086","protocol":"http","database":"EnergyBridgeDB","name":"","usetls":false,"tls":"","influxdbVersion":"1.8-flux","url":"http://influxdb:8086","rejectUnauthorized":true}]