ESP32 Classic MQTT Publisher & Home Assistant

Started by Graham, February 13, 2020, 05:58:57 PM

Previous topic - Next topic

Graham

I created a wiki for my ESP32 ClassicMQTT publisher project on github that includes an example on how to setup HomeAssistant to view the classics data.
Also have some more info on how to setup the beta version of the Android MQTT subscriber app.

https://github.com/graham22/ClassicMQTT/wiki

Let me know if you have any comments or suggestion...

Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Thanks Graham - I will dig out my esp32 and give it a try sometime soon.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

#2
I found my esp32 . First I tried the flash tool but couldn't get it to work. Then I watched Andreas Spies video on how to set up platformio and got that set up and figured out how to import your file to it. And how to set up the board etc. Same error as with other flash tool. I found that I needed to hold down the boot button on my esp32 while flashing it ( or to add a capacitor https://randomnerdtutorials.com/solved-failed-to-connect-to-esp32-timed-out-waiting-for-packet-header/) .
I held down the button and flashed it with platformio and it worked.
I needed to reread some forum posts to find the password for 192.168.4.1 access point which showed up on wifi ( it was ClassicMQTT or whatever that post said ) . So I got into the menu but it wanted me to put in mqtt . I had signed up for DIoTY using Google and I get a page that says they email me a password but I never got one - wondering if I had signed up for this last time I tried your other classic mqtt project ?  I think I had used a different mqtt that you had suggested at that time. I emailed for password reminder but didn't hear anything back yet.
I can only connect to it using AP on 192.168.4.1  but can't get any farther than that - I mean I thought I saw it connect to my access point once but I can't connnect to it that way  . Am I supposed to be able to connect to it via router instead of directly AP way ?
I did get the android app installed on my phone but don't think I can go any farther until I find out the password from DIoTY .
Tomorrow is another day !
update - I was able to connect to it now via my router - I see that it is the same setup file as connecting to it as AP.
I found my DIoTY password from last year . I put it in both android classic and the esp32 along with all the other settings. So far nothing is coming in on the android app - maybe still a password issue . Or maybe some setting in my android phone is holding it back. I am pretty sure the esp32 is connected to the Classic because I can't get into the it with Local Status .
Getting late here so will give it a try again tomorrow.
I don't see this
"The navigation bar should initially show "MQTT Broker", click it to connect."
I looked at the youtube setup video for Home Assistant - that is really a job for a fresh day !

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

mcsarge

Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Next, I will implement the MQTT stuff so that I can subscribe and publish the data.

I notice in your code that there are certain values that you only get one time, in my decoding, it takes no more time to decode them to not decode them, so I just get them every time, I will setup the publish stuff in the way that you codes does it so that the result of what is published will be the same as in Classic MQTT.

Matt
Off Grid Island in Ontario Canada (Beaverstone Bay)
Primary: Classic 150 + wbjr; 3s2p HES 270watt
Winter: SolarBoost 50 MPPT (into wbjr); 2 x Sharp NE-80EJEA 80watt
Pack: 4s2p ROLLS S6-460AGM 6V for 24V pack
Inverter/Charger: Trace DR2424
Call Sign: KG4EUF

ClassicCrazy

I should not have to open a port on my router for the MQTT to work - right ?

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

mcsarge

Larry,

No, your ESP32 should reach _out_ to the MQTT server if you are using one out on the internet. If you setup your own MQTT server behind a firewall, then you may need to open a port - but I do not think that is what you are doing.

Matt
Off Grid Island in Ontario Canada (Beaverstone Bay)
Primary: Classic 150 + wbjr; 3s2p HES 270watt
Winter: SolarBoost 50 MPPT (into wbjr); 2 x Sharp NE-80EJEA 80watt
Pack: 4s2p ROLLS S6-460AGM 6V for 24V pack
Inverter/Charger: Trace DR2424
Call Sign: KG4EUF

ClassicCrazy

Quote from: mcsarge on February 18, 2020, 12:54:57 PM
Larry,

No, your ESP32 should reach _out_ to the MQTT server if you are using one out on the internet. If you setup your own MQTT server behind a firewall, then you may need to open a port - but I do not think that is what you are doing.

Matt
yeah that is what I thought . Not sure why my data isn't getting out or back in . I have checked and rechecked all the settings. I may try a different cloud mqtt server tonight and see what happens.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

mcsarge

Larry,

I have an MQTT server setup on Azure, you are welcome to give it a try. I can PM you the particulars.

Matt
Off Grid Island in Ontario Canada (Beaverstone Bay)
Primary: Classic 150 + wbjr; 3s2p HES 270watt
Winter: SolarBoost 50 MPPT (into wbjr); 2 x Sharp NE-80EJEA 80watt
Pack: 4s2p ROLLS S6-460AGM 6V for 24V pack
Inverter/Charger: Trace DR2424
Call Sign: KG4EUF

ClassicCrazy

Quote from: mcsarge on February 18, 2020, 03:01:18 PM
Larry,

I have an MQTT server setup on Azure, you are welcome to give it a try. I can PM you the particulars.

Matt
sure I will give it a try - PM me what I need to know
thanks
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

I was monitoring the serial ouput of the Esp32 .
It seems to be connecting to the Classic okay .
But I see this error when it tries to connect to the MQTT server

connectToMqtt(): Connecting to MQTT...
[E][AsyncTCP.cpp:729] connect(): error: -16

Next I was going to try Matt's server on the android Classic app but the address of his server is too long to fit in the space for it in the Classic App

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

Thanks to Matt's server - it is working now . Esp32 is talking to the Classic and I can monitor it on the Android  Classic Monitor app. There must have been some kind of problem using that other mqtt server I tried at first.
I will try loading up Home Assistant stuff next .

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: mcsarge on February 18, 2020, 09:25:43 AM
Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Matt

Yes, if someone already has a raspberry Pi setup for home assistant then adding the python implementation for the ClassicMQTT publisher would be a lot simpler than having to use a ESP32.
I created a folder in the github project under code https://github.com/graham22/ClassicMQTT/tree/master/code/Python.
If we use a 'docker container' implementation then it could be added to HASS community AddOns https://developers.home-assistant.io/docs/en/hassio_addon_index.html

Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

Graham

Quote from: ClassicCrazy on February 19, 2020, 12:15:35 AM
Thanks to Matt's server - it is working now . Esp32 is talking to the Classic and I can monitor it on the Android  Classic Monitor app. There must have been some kind of problem using that other mqtt server I tried at first.
I will try loading up Home Assistant stuff next .

Larry

Congrats Larry
I often use the MQTTfx tool https://mqttfx.jensd.de/ and the MQTTExplorer http://mqtt-explorer.com/ to see/debug the MQTT data
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

mcsarge

Thanks Graham,

As soon as I have the bones of a decent implementation I will post. Right now I only have what amounts to a script that connects and reads and decodes each of the address locations. I need to add JSON formatting of the data and the MQTT publish/subscribe stuff and then make it run continuously.

Question: I am not using async to read the MODBUS data from the Classic, it seems to me since the Classic can only talk to one device at a time, there might not be much of a delay so maybe async is not necessary. Have you found that it is necessary? In my test script I get the data very fast and I am connecting to the Classic over ngrok on a cellular link.

Matt

Quote from: Graham on February 19, 2020, 07:11:08 AM
Quote from: mcsarge on February 18, 2020, 09:25:43 AM
Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Matt

Yes, if someone already has a raspberry Pi setup for home assistant then adding the python implementation for the ClassicMQTT publisher would be a lot simpler than having to use a ESP32.
I created a folder in the github project under code https://github.com/graham22/ClassicMQTT/tree/master/code/Python.
If we use a 'docker container' implementation then it could be added to HASS community AddOns https://developers.home-assistant.io/docs/en/hassio_addon_index.html

Graham.
Off Grid Island in Ontario Canada (Beaverstone Bay)
Primary: Classic 150 + wbjr; 3s2p HES 270watt
Winter: SolarBoost 50 MPPT (into wbjr); 2 x Sharp NE-80EJEA 80watt
Pack: 4s2p ROLLS S6-460AGM 6V for 24V pack
Inverter/Charger: Trace DR2424
Call Sign: KG4EUF

Graham

Quote from: mcsarge on February 19, 2020, 07:34:14 AM
Question: I am not using async to read the MODBUS data from the Classic, it seems to me since the Classic can only talk to one device at a time, there might not be much of a delay so maybe async is not necessary. Have you found that it is necessary? In my test script I get the data very fast and I am connecting to the Classic over ngrok on a cellular link.


I used async in the ESP32 so that a tcp call to modbus would not block the web server and OTA code, this might not be as important in your case...
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s