A Forum run by Enthusiasts of MidNite Solar

The Open Source software/hardware corner => General info => Topic started by: Graham on February 13, 2020, 05:58:57 PM

Title: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 13, 2020, 05:58:57 PM
I created a wiki for my ESP32 ClassicMQTT publisher project on github that includes an example on how to setup HomeAssistant to view the classics data.
Also have some more info on how to setup the beta version of the Android MQTT subscriber app.

https://github.com/graham22/ClassicMQTT/wiki (https://github.com/graham22/ClassicMQTT/wiki)

Let me know if you have any comments or suggestion...

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 13, 2020, 06:48:31 PM
Thanks Graham - I will dig out my esp32 and give it a try sometime soon.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 17, 2020, 09:29:09 PM
I found my esp32 . First I tried the flash tool but couldn't get it to work. Then I watched Andreas Spies video on how to set up platformio and got that set up and figured out how to import your file to it. And how to set up the board etc. Same error as with other flash tool. I found that I needed to hold down the boot button on my esp32 while flashing it ( or to add a capacitor https://randomnerdtutorials.com/solved-failed-to-connect-to-esp32-timed-out-waiting-for-packet-header/) .
I held down the button and flashed it with platformio and it worked.
I needed to reread some forum posts to find the password for 192.168.4.1 access point which showed up on wifi ( it was ClassicMQTT or whatever that post said ) . So I got into the menu but it wanted me to put in mqtt . I had signed up for DIoTY using Google and I get a page that says they email me a password but I never got one - wondering if I had signed up for this last time I tried your other classic mqtt project ?  I think I had used a different mqtt that you had suggested at that time. I emailed for password reminder but didn't hear anything back yet.
I can only connect to it using AP on 192.168.4.1  but can't get any farther than that - I mean I thought I saw it connect to my access point once but I can't connnect to it that way  . Am I supposed to be able to connect to it via router instead of directly AP way ?
I did get the android app installed on my phone but don't think I can go any farther until I find out the password from DIoTY .
Tomorrow is another day !
update - I was able to connect to it now via my router - I see that it is the same setup file as connecting to it as AP.
I found my DIoTY password from last year . I put it in both android classic and the esp32 along with all the other settings. So far nothing is coming in on the android app - maybe still a password issue . Or maybe some setting in my android phone is holding it back. I am pretty sure the esp32 is connected to the Classic because I can't get into the it with Local Status .
Getting late here so will give it a try again tomorrow.
I don't see this
"The navigation bar should initially show "MQTT Broker", click it to connect."
I looked at the youtube setup video for Home Assistant - that is really a job for a fresh day !

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 18, 2020, 09:25:43 AM
Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Next, I will implement the MQTT stuff so that I can subscribe and publish the data.

I notice in your code that there are certain values that you only get one time, in my decoding, it takes no more time to decode them to not decode them, so I just get them every time, I will setup the publish stuff in the way that you codes does it so that the result of what is published will be the same as in Classic MQTT.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 18, 2020, 10:46:58 AM
I should not have to open a port on my router for the MQTT to work - right ?

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 18, 2020, 12:54:57 PM
Larry,

No, your ESP32 should reach _out_ to the MQTT server if you are using one out on the internet. If you setup your own MQTT server behind a firewall, then you may need to open a port - but I do not think that is what you are doing.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 18, 2020, 02:19:55 PM
Quote from: mcsarge on February 18, 2020, 12:54:57 PM
Larry,

No, your ESP32 should reach _out_ to the MQTT server if you are using one out on the internet. If you setup your own MQTT server behind a firewall, then you may need to open a port - but I do not think that is what you are doing.

Matt
yeah that is what I thought . Not sure why my data isn't getting out or back in . I have checked and rechecked all the settings. I may try a different cloud mqtt server tonight and see what happens.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 18, 2020, 03:01:18 PM
Larry,

I have an MQTT server setup on Azure, you are welcome to give it a try. I can PM you the particulars.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 18, 2020, 03:30:45 PM
Quote from: mcsarge on February 18, 2020, 03:01:18 PM
Larry,

I have an MQTT server setup on Azure, you are welcome to give it a try. I can PM you the particulars.

Matt
sure I will give it a try - PM me what I need to know
thanks
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 18, 2020, 10:47:33 PM
I was monitoring the serial ouput of the Esp32 .
It seems to be connecting to the Classic okay .
But I see this error when it tries to connect to the MQTT server

connectToMqtt(): Connecting to MQTT...
[E][AsyncTCP.cpp:729] connect(): error: -16

Next I was going to try Matt's server on the android Classic app but the address of his server is too long to fit in the space for it in the Classic App

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 19, 2020, 12:15:35 AM
Thanks to Matt's server - it is working now . Esp32 is talking to the Classic and I can monitor it on the Android  Classic Monitor app. There must have been some kind of problem using that other mqtt server I tried at first.
I will try loading up Home Assistant stuff next .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 19, 2020, 07:11:08 AM
Quote from: mcsarge on February 18, 2020, 09:25:43 AM
Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Matt

Yes, if someone already has a raspberry Pi setup for home assistant then adding the python implementation for the ClassicMQTT publisher would be a lot simpler than having to use a ESP32.
I created a folder in the github project under code https://github.com/graham22/ClassicMQTT/tree/master/code/Python (https://github.com/graham22/ClassicMQTT/tree/master/code/Python).
If we use a 'docker container' implementation then it could be added to HASS community AddOns https://developers.home-assistant.io/docs/en/hassio_addon_index.html (https://developers.home-assistant.io/docs/en/hassio_addon_index.html)

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 19, 2020, 07:16:21 AM
Quote from: ClassicCrazy on February 19, 2020, 12:15:35 AM
Thanks to Matt's server - it is working now . Esp32 is talking to the Classic and I can monitor it on the Android  Classic Monitor app. There must have been some kind of problem using that other mqtt server I tried at first.
I will try loading up Home Assistant stuff next .

Larry

Congrats Larry
I often use the MQTTfx tool https://mqttfx.jensd.de/ (https://mqttfx.jensd.de/) and the MQTTExplorer http://mqtt-explorer.com/ (http://mqtt-explorer.com/) to see/debug the MQTT data
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 19, 2020, 07:34:14 AM
Thanks Graham,

As soon as I have the bones of a decent implementation I will post. Right now I only have what amounts to a script that connects and reads and decodes each of the address locations. I need to add JSON formatting of the data and the MQTT publish/subscribe stuff and then make it run continuously.

Question: I am not using async to read the MODBUS data from the Classic, it seems to me since the Classic can only talk to one device at a time, there might not be much of a delay so maybe async is not necessary. Have you found that it is necessary? In my test script I get the data very fast and I am connecting to the Classic over ngrok on a cellular link.

Matt

Quote from: Graham on February 19, 2020, 07:11:08 AM
Quote from: mcsarge on February 18, 2020, 09:25:43 AM
Graham,

As you may recall, I have been working on a Python client to do what the ClassicMQTT does - so I can run it on Pi or other SBC. I wanted to give you an update. I have successfully created a decoder that is able to decode all the packets of data that are returned by each of the addresses you query in the ESP32 code. I can post that code if you want, but I was thinking maybe we should set something up in Git to capture this - maybe a python directory or something?

Matt

Yes, if someone already has a raspberry Pi setup for home assistant then adding the python implementation for the ClassicMQTT publisher would be a lot simpler than having to use a ESP32.
I created a folder in the github project under code https://github.com/graham22/ClassicMQTT/tree/master/code/Python (https://github.com/graham22/ClassicMQTT/tree/master/code/Python).
If we use a 'docker container' implementation then it could be added to HASS community AddOns https://developers.home-assistant.io/docs/en/hassio_addon_index.html (https://developers.home-assistant.io/docs/en/hassio_addon_index.html)

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 19, 2020, 07:46:23 AM
Quote from: mcsarge on February 19, 2020, 07:34:14 AM
Question: I am not using async to read the MODBUS data from the Classic, it seems to me since the Classic can only talk to one device at a time, there might not be much of a delay so maybe async is not necessary. Have you found that it is necessary? In my test script I get the data very fast and I am connecting to the Classic over ngrok on a cellular link.


I used async in the ESP32 so that a tcp call to modbus would not block the web server and OTA code, this might not be as important in your case...
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 19, 2020, 08:55:41 PM
Oh right, that makes sense. Running inside Docker or directly from the Pi, this would be less of an issue.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 19, 2020, 09:17:02 PM
I got my Raspberry Pi 3B+ running - I had earlier installed Docker and some other programs on it using the Andreas Spies video about it. One of the options on the installer is HAAS IO so I ran that awhile ago.
I am still trying to figure out how to open up and use all these programs and features.
I also have Mosquito MQTT on it so I want to try to use that with the esp32 and Classic for my local network use here.
There is a lot to learn and setup.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 19, 2020, 10:04:38 PM
Larry,

The cool thing about having the MQTT outside your network is that you will not have to poke a hole in your firewall for the app to reach into. But setting up Mosquito is super easy, just remember you need to add users.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 19, 2020, 10:36:09 PM
ha ha - I wouldn't call any of this stuff super easy !  Maybe when I get my head wrapped around it a little bit more. At least I am out of the novice linux and raspberry pi stage so  I know what is getting talked about and how to do stuff either by ssh into pi or to vnc into it for full web interface.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 26, 2020, 12:15:34 AM
I did get Grahams esp32 code working to his Android Classic monitor using MQTT server via Matt.
But then I decided I wanted to install Mosquitto MQTT Broker on my own Raspberry Pi and use that locally so I could learn how that works and also try to do some other MQTT stuff around here maybe via node red.
So I downloaded the Andreas Spiess recommended Docker and other installer by the other Graham from South Africa  - everything installed okay - I can get Portainer to run on the pi chrome browser and Node Red too by just  going to their web interfaces.
But I have spent many nights reading up or trying to figure out the Mosquitto stuff - watched lots of videos . I  find that other  instructional videos won't work unless I start over with their naming systems from scratch.
I think everyone assumes that  everyone has some knowledge of the most confusing file naming and location and linux schemes imaginable. I admit defeat for now. It seems there is some simple step to putting a files somewhere else on the Pi to make mosquitto work but I just don't get it and nothing I have tried works. I get halfway into someones detailed video before they leave me in the dust and my eyes glaze over  and I am convinced they are just talking gibberish and having a laugh that someone out there wasted their time listening to their meaningless BS.
So thanks Graham and Matt - your explanations and your stuff works. For the rest of it I may have to wipe the pi and docker and start out fresh and not use the predownloaded method  Andreas explains because they assume I probably understand all the untold steps and I don't !
But I did put in a lot of hours trying.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 26, 2020, 07:46:07 AM
Sorry to hear your having some frustration with the setup
I would just install home assistant on the Raspberry PI and use the add-on store within the hassio/dashboard to install Node-Red and the mosquito broker. The HassIO image comes pre-configured with everything you need like docker etc...

The Mosquito broker is in the "Official add-ons" repository and NodeRed is in the "Community Add-ons" repository

https://www.home-assistant.io/getting-started/ (https://www.home-assistant.io/getting-started/)


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 26, 2020, 11:03:38 AM
Hello Fellows.

I am attempting to read from the Classic registers starting at 16385, I would like to read the app_rev and the net_rev. When I try to read those over the MODBUS I get an error. Any suggestions? That register number is alot higher than the others, so maybe it is just not there?

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on February 26, 2020, 11:47:47 AM
Quote from: Graham on February 26, 2020, 07:46:07 AM
Sorry to hear your having some frustration with the setup
I would just install home assistant on the Raspberry PI and use the add-on store within the hassio/dashboard to install Node-Red and the mosquito broker. The HassIO image comes pre-configured with everything you need like docker etc...

The Mosquito broker is in the "Official add-ons" repository and NodeRed is in the "Community Add-ons" repository

https://www.home-assistant.io/getting-started/ (https://www.home-assistant.io/getting-started/)

Graham.

Thanks Graham- I will give that a try !  I didn't realize it was something ready to go as a flash and maybe when i tried it before I was doing it on top of other install .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 26, 2020, 01:45:27 PM
Graham,

I have been able to collect the MAC address and I can send it on in the "info" package - do you have an identifier that I should use to send it to the app, or are you not yet gathering that? If you have not done it yet, I am using the identifier in the JSON of "mac".

I can add similar code the ESP32 code if you want.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 27, 2020, 07:24:09 AM
Quote from: mcsarge on February 26, 2020, 11:03:38 AM
Hello Fellows.

I am attempting to read from the Classic registers starting at 16385, I would like to read the app_rev and the net_rev. When I try to read those over the MODBUS I get an error. Any suggestions? That register number is alot higher than the others, so maybe it is just not there?

Matt

I have a modbus tool I wrote a few years ago that might be useful, it's a bit buggy but gets the job done
https://github.com/graham22/ModbusTool (https://github.com/graham22/ModbusTool)

It includes a master & Slave app, you can connect to the classic using the master, once you read some registers, you can export the data and load it into the slave app to emulate the classic then you can manually change some register values to see how it affects the android app or MQTT publisher...

Attached is a snip of the Modbus Master having read 4 bytes at 16386 from the classic


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 27, 2020, 07:56:33 AM
Quote from: mcsarge on February 26, 2020, 01:45:27 PM
Graham,

I have been able to collect the MAC address and I can send it on in the "info" package - do you have an identifier that I should use to send it to the app, or are you not yet gathering that? If you have not done it yet, I am using the identifier in the JSON of "mac".

I can add similar code the ESP32 code if you want.

Matt

unfortunately the android subscriber is not expecting the mac address in the ChargeControllerTransfer.java class.

the MQTT version of the android app needs some re-factoring since it currently can only handle one classic

here is my todo list for the app

1. remove the MQTT publisher code since it makes more sense to use a Rpi or a esp32 for this
2. move the root topic setting out of the config and put it into the nav bar + so you can add more than one just like adding an IP address/port in the regular app version
3. Update the info transfer to include the missing data

#2 is the most difficult since the app would have to subscribe to multiple root topics to sum the values used in the system view...


Graham.

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 27, 2020, 02:30:38 PM
OK,

I was able to get the app_rev and net_rev read - I looked at your code in the Classic Android App and noticed you were reading not at 16384 but at the location of the 2 values you are interested in. Once I did that I could collect the values no problem.

I now have the code running on my Pi on the island pumping data to an MQTT on azure.

It seems to work on my copy of the Android app...

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 27, 2020, 03:50:03 PM
OK,

I created a pull request so you can grab the python version.

It requires python3, pymodbus and paho-mqtt.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 27, 2020, 07:39:47 PM
Give me a little more time on this, I am working out some issues with when mqtt disconnects and reconnects.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 02, 2020, 12:07:38 AM
I did get the Home Assistant flashed on the Raspberry Pi here.
I made some progress but very slow progress figuring out Home Assistant  and it isn't for lack of spending time watching tutorials.
Biggest issues I find
They have made so many changes to Home Assistant that none of the old tutorials make a lot of sense with new versions.
Some of the things have been renamed to make it even more confusing. For example the add on File Editor used to be called Configurator .
Many of the detailed instructions don't work - for example I installed add on Mosquitto Broker and followed the steps - but they show a screen in the install or setup for configuring it - but that setup screen just does not ever show up.
I tried adding custom cards to the UI- sort of - watched tutorials but some of those guys don't show a blown up screen of what they are doing and go so fast and are working with the old version , etc . 
Nothing I try ever seems to work - such as just paste something to the Yaml config but then it has all kinds of errors showing up - spacing , indents , this, that, and the other thing ! 
This Home Assistant program just is not intuitive - maybe some day it will be.
I think I will go back to trying to figure out Node Red - at least that program isn't all over the map like Home Assistant.
But as I said before - if I was just doing the basic project in this forum topic- getting an Esp32 programmed and publishing to MQTT broker and then using Android with Classic Monitoring  app to subscribe to that and display the Classic data - yes got that done and that works !

Larry

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 02, 2020, 02:07:15 PM
Graham,

OK, I have the nearly final, ready to be public version all checked in and ready.

Thanks,

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 02, 2020, 02:23:32 PM
Quote from: mcsarge on March 02, 2020, 02:07:15 PM
Graham,

OK, I have the nearly final, ready to be public version all checked in and ready.

Thanks,

Matt

Ok, I merged it

ordered another RPI so I can try it out and start to learn python...


Graham.

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 02, 2020, 03:04:39 PM
You can actually run Python on a Mac or a Windoze PC, you just have to install it. I like to us VSCode to develop in, and from there you can launch the program easily if all is installed. While developing. I used ngrok to get to port 502 of my Classic up on the island, but now that it is running on my pi up there, I use ngrok to get to MQTT installed on the Pi - and I use 0.tcp.ngrok.io and that port in the Android App to look at the data.

Now that I have the infrastructure installed, I am going to work on a large LCD or LED display using an ESP32 for the wall of the cabin that will show some of the data from the Classic like battery voltage, WhizBangJr current and SOC, etc. It will have a button that will send a "wake" to get data more often, but for most applications, updates every 5 minutes (snoozing) will be just fine.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 02, 2020, 11:07:19 PM
It will be interesting to see your display when you get it going Matt.
That is what I have been wanting to do here - just be able to see certain selected data from the Classic on a small LED display on the wall.
I know I could hang the MNGP up - but that is too much data and have to do bunch of button pushes.
I guess if I had to pick a few vital clues it would be
Charge state ,SOC,  system amps, target voltage ( temperature compensated voltage) and the battery voltage. Oh one more might be the Absorb timer . Those are the main things I usually look for at a glance .
Oh and if you want to make a big LED display here is an idea https://youtu.be/fz2QAV9z_o8

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 03, 2020, 07:21:23 AM
Quote from: mcsarge on March 02, 2020, 03:04:39 PM
You can actually run Python on a Mac or a Windoze PC, you just have to install it. I like to us VSCode to develop in, and from there you can launch the program easily if all is installed. While developing. I used ngrok to get to port 502 of my Classic up on the island, but now that it is running on my pi up there, I use ngrok to get to MQTT installed on the Pi - and I use 0.tcp.ngrok.io and that port in the Android App to look at the data.


Is your MQTT broker local on the Pi or in Azure (or any other cloud based broker)?

+ we could "Containerize" the python app using docker that would allow it to run anywhere
https://www.wintellect.com/containerize-python-app-5-minutes/ (https://www.wintellect.com/containerize-python-app-5-minutes/)


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 03, 2020, 04:45:40 PM
Pretty funny you say that  as I am working on a Docker container that has Python3 on it and can run the App and will also have Mosquito running on it.

I have used both the Azure MQTT (I still have it up and running) but currently I have it using a Mosquito running on the Pi.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 05, 2020, 10:47:14 PM
Quote from: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt
Let me know if you want me to test it out sometime.
I have raspberry pi 3 and can reflash it .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 06, 2020, 08:00:15 AM
Quote from: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt

Thanks Matt, I merged the code on github.
I'll give it a try when I get my new raspberry Pi 4


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 06, 2020, 09:17:18 AM
Quote from: ClassicCrazy on March 05, 2020, 10:47:14 PM
Quote from: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt
Let me know if you want me to test it out sometime.
I have raspberry pi 3 and can reflash it .

Larry

Larry,

Have at it, it is particular to Stretch, but a small change can support other distros of Raspbarian - There is an issue with running Docker on the latest version of Rasparian, so I am sticking to stretch.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 12:19:11 AM
Here is what I have done so far
I am using Pi3B+  model
Installed latest Raspian Buster ( stretch version not on pi download page so giving the Buster version a try)
set up pi for using ssh, vnc, using the whole sd card , and locale
updated and upgraded pi
Installed Docker and Docker compose according to
https://dev.to/rohansawant/installing-docker-and-docker-compose-on-the-raspberry-pi-in-5-simple-steps-3mgl
Noticed that some people in comments on above link said they had problems because default for docker compose was python 2
I followed directions on below link to change default to docker 3
https://learn.sparkfun.com/tutorials/python-programming-tutorial-getting-started-with-the-raspberry-pi/configure-your-pi
I reinstalled docker compose just to make sure it was correct
Now on to Matt's directions in Read Me file
install pymodbus on pi ( think I installed it for both python 2 and python 3 ?) using info on next link
https://www.raspberrypi.org/forums/viewtopic.php?t=251917
install paho mqtt using this command    pip install paho-mqtt
this was the paho mqtt info i used
http://www.steves-internet-guide.com/into-mqtt-python-client/

From Matts instructions
Install docker and docker-compose on your Rasberry Pi - look this up on the web and follow the instructions.( done as above)

Copy the repostory (if you understand git, you can get it that way too)
wget https://github.com/graham22/ClassicMQTT/archive/master.zip ( ran this )
Extract the zip file:
unzip master.zip ( ran this )
Change directory
cd ./ClassicMQTT/code/Python ( this step gets me lost - I get this error )
-bash: cd: ./ClassicMQTT/code/Python: No such file or directory
I think this step should say  cd ClassicMQTT-master/code/Python   ( is this correct ?)
Then I did
nano .env  ( to create and pasted in contents below with my classic ip address , etc ) 

Create the .env file to look like this:
CLASSIC_HOST=<IP address or URL>
CLASSIC_PORT=<Port usually 502>
CLASSIC_ROOT=<The MQTT Root, usually ClassicMQTT>
USER_ID=ClassicPublisher
USER_PASS=ClassicPub123


User docker-compose to start up the both the mosquitto and the script.
docker-compose -f classic_mqtt_compose.yml up

did the command above it is seems to work - no errors anyway - I saw it loading up : Downloaded newer image for python:3.7.6-slim-stretch
Should I change something in the code to download buster since I am using that and not stretch ?
I am not exactly sure the next step of what I put in my android to subscribe to this
I tried subscribing to 172.18.0.3 on port 1883 but I don't see it . My pi is plugged into the router - no wifi connected yet - and my android is wifi to my router . 
Screen shot below of my pi data
Calling it a night now but I think I had some success for not knowing much about what I am doing on Linux stuff.
Little things like that cd command would throw me off for good but tonight I think I figured out what to do .

Larry


Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 07, 2020, 08:21:39 AM
Quote from: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt

Still waiting for my argon case for my raspberry pi 4 (https://www.argon40.com/catalog/product/view/id/52/s/argon-one-raspberry-pi-4-case/category/4/ (https://www.argon40.com/catalog/product/view/id/52/s/argon-one-raspberry-pi-4-case/category/4/))
so I decided to try this using docker desktop on Windows 10 pro using the following yaml file that uses my existing mosquito MQTT  broker.

command:

docker-compose -f classic_mqtt_compose_nomosquito.yml up

classic_mqtt_compose_nomosquito.yml content:

version: "3"
services:
  classic_mqtt:
    image: classic_mqtt
    build: .
    environment:
      - LOGLEVEL=DEBUG
    networks:
      - localnet
    command: "--classic 192.168.86.37 --classic_port 502 --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --user HASS --pass XXXX"
networks:
  localnet:

I was then able to view the data using the MQTT version of my android app.
Docker desktop for Windows 10 PRO (doesn't install on Windows 10 Home edition) : https://hub.docker.com/editions/community/docker-ce-desktop-windows (https://hub.docker.com/editions/community/docker-ce-desktop-windows)

Also tried it on Ubuntu 19.04
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 07, 2020, 08:24:52 AM
Congratulations (Larry)!

You are up and running (I can see it is getting info from your Classic) To use the flow, go into Graham's app and configure it to be an MQTT Subscriber and set the IP address of the MQTT server your Pi's IP address and port 1883. It should connect and off you go.

If your system is behind a firewall, you can either port forward that port in your router or install ngrok and forward the TCP port with the following command:
./ngrok 1883

So the beauty of using docker is that you do not need to install pymodbus or paho-mqtt. or even python at all! It all gets installed in the container for you and saved (I have directions for 2 modes, so I will fix that to be more clear). I will also fix the cd instruction (I forgot that the "-master" gets in there. You can change your python in the "Dockerfile" to be FROM python:3.7.6-slim-buster if you want but since they rely on the same architecture, it does not make much difference.

Thanks for testing!

I am now throwing together a website written in python using flask and socketio to connect to the MQTT server and display some of the readings like Battery Voltage, SOC, Current, etc. Once I get the simple monitor done, I will add in code to save the readings every 5 minutes in a database so I can show charts that cover readings over a period of time in chart fashion.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 07, 2020, 08:29:02 AM
Quote from: Graham on March 07, 2020, 08:21:39 AM
Quote from: mcsarge on March 05, 2020, 12:20:30 PM
Graham,

I have checked in the docker and docker-compose for the tool and created a pull request. Pretty slick, if I do say so myself.

So all you need is a clean Raspberry Pi, install docker and docker-compose, get the Python folder onto the Pi, create your .env file and run docker-compose and you are in business. I documented all this in the README.

Matt

Awesome!

I will add the no_mosquitto yaml file in for people who have mqtt already installed and running.

Matt

Still waiting for my argon case for my raspberry pi 4 (https://www.argon40.com/catalog/product/view/id/52/s/argon-one-raspberry-pi-4-case/category/4/ (https://www.argon40.com/catalog/product/view/id/52/s/argon-one-raspberry-pi-4-case/category/4/))
so I decided to try this using docker desktop on Windows 10 pro using the following yaml file that uses my existing mosquito MQTT  broker.

command:

docker-compose -f classic_mqtt_compose_nomosquito.yml up

classic_mqtt_compose_nomosquito.yml content:

version: "3"
services:
  classic_mqtt:
    image: classic_mqtt
    build: .
    environment:
      - LOGLEVEL=DEBUG
    networks:
      - localnet
    command: "--classic 192.168.86.37 --classic_port 502 --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --user HASS --pass XXXX"
networks:
  localnet:

I was then able to view the data using the MQTT version of my android app.
Docker desktop for Windows 10 PRO (doesn't install on Windows 10 Home edition) : https://hub.docker.com/editions/community/docker-ce-desktop-windows (https://hub.docker.com/editions/community/docker-ce-desktop-windows)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 10:46:43 AM
Matt ,
I don't have a firewall - but my android is still not connecting
I set the Grahams Classic Monitoring to
Mqtt subscriber
Mqtt broker host name to the router ip address that the pi has which is 192.168.3.29
mqtt broker port to 1883

I used the same mqtt user and passwords and root topic as you had in examples.

But I still don't connect with android.

Do I have to edit the mqtt host file  somewhere ?- - it says 127.0.0.1 is default in the code file
Also wondering if I should start out fresh again in case those extra copies of pymod and mosquitto I installed outside of docker are messing anything up ?

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 07, 2020, 10:52:03 AM
Matt,
I posted a pull request on my ClassicMQTT github page for you to review,
I want to add a "will" so that the android app can detect when the publisher goes offline so it can clear the gauges

https://github.com/graham22/ClassicMQTT/pull/6 (https://github.com/graham22/ClassicMQTT/pull/6)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 07, 2020, 11:01:59 AM
Quote from: ClassicCrazy on March 07, 2020, 10:46:43 AM
Matt ,
I don't have a firewall - but my android is still not connecting
I set the Grahams Classic Monitoring to
Mqtt subscriber
Mqtt broker host name to the router ip address that the pi has which is 192.168.3.29
mqtt broker port to 1883

I used the same mqtt user and passwords and root topic as you had in examples.

But I still don't connect with android.

Do I have to edit the mqtt host file  somewhere ?- - it says 127.0.0.1 is default in the code file
Also wondering if I should start out fresh again in case those extra copies of pymod and mosquitto I installed outside of docker are messing anything up ?

Larry

Larry,
have you tried using the MQTT.fx tool see if the data is being published

you can subscribe to "ClassicMQTT/#" to get all messages
you can also publish a "WAKE" command to get the publisher to send readings


Graham.


https://mqttfx.jensd.de/index.php/download (https://mqttfx.jensd.de/index.php/download)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 11:05:38 AM
Matt
Should this .env file have another entry for mqtt broker address ?
Guess I could try it and see what happens

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 11:07:40 AM
Quote from: Graham on March 07, 2020, 11:01:59 AM
Quote from: ClassicCrazy on March 07, 2020, 10:46:43 AM
Matt ,
I don't have a firewall - but my android is still not connecting
I set the Grahams Classic Monitoring to
Mqtt subscriber
Mqtt broker host name to the router ip address that the pi has which is 192.168.3.29
mqtt broker port to 1883

I used the same mqtt user and passwords and root topic as you had in examples.

But I still don't connect with android.

Do I have to edit the mqtt host file  somewhere ?- - it says 127.0.0.1 is default in the code file
Also wondering if I should start out fresh again in case those extra copies of pymod and mosquitto I installed outside of docker are messing anything up ?

Larry

Larry,
have you tried using the MQTT.fx tool see if the data is being published

you can subscribe to "ClassicMQTT/#" to get all messages
you can also publish a "WAKE" command to get the publisher to send readings


Graham.


https://mqttfx.jensd.de/index.php/download (https://mqttfx.jensd.de/index.php/download)

I will give that tool a try Graham

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 11:20:58 AM
I think adding that MQTT broker line with my router ip address to the .env file did the trick because now the Android connected up and is reading all the data .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 07, 2020, 02:34:52 PM
Larry,

When the tool is started in the compose file, I pass in "mosquitto" for the --mqtt parameter that should specify the virtual hostname mosquitto in the compose file. Not sure why it would not find it living there. If you add a parameter to the .env file, I would not use it unless you add that to the command.

If you zip up your Python directory and send it to me I can take a better look matthew dot c dot sargent at gmail etc.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 07, 2020, 02:53:05 PM
BTW,

I have been using MQTT Explorer and it works well for me.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 07, 2020, 07:11:29 PM
Quote from: mcsarge on March 07, 2020, 02:53:05 PM
BTW,

I have been using MQTT Explorer and it works well for me.

Matt

I just got the MQTT Explorer running and connected.
I did not have success with the MQTT FX tool

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 08, 2020, 08:58:02 PM
I just tested  - when the Pi is connected to Classic and publishing to the Classic Android app - if I connect up to the Classic using the Midnite Local Status app - that app will connect and take over and the Android app looses its data. As soon as I close the Midnite Local Status app - the Android app responds right away with the data from the Classic again.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 09, 2020, 07:29:44 AM
This is expected behavior because the Python program connects to the modbus port, gets the data, and then disconnects every 5 seconds or 5 minutes. The localapp connects to the modbus port and then doesn't disconnect until you stop running the program
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 09, 2020, 01:12:10 PM
Quote from: mcsarge on March 09, 2020, 07:29:44 AM
This is expected behavior because the Python program connects to the modbus port, gets the data, and then disconnects every 5 seconds or 5 minutes. The localapp connects to the modbus port and then doesn't disconnect until you stop running the program

I like how this works - no problems when I want to connect on Midnite Local app .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 10, 2020, 03:53:53 PM
Graham,

I have updated the tool to use timeloop - a great library for timed executions and such. I have also re-written the readme totally to describe the 3 wys to install the tool. I will start a pull-request.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 10, 2020, 05:05:55 PM
Oops, I edited the Wiki on your repo, so the wiki does not match the code until you do the Pull. Sorry.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 10, 2020, 05:07:10 PM
Quote from: mcsarge on March 10, 2020, 03:53:53 PM
Graham,

I have updated the tool to use timeloop - a great library for timed executions and such. I have also re-written the readme totally to describe the 3 wys to install the tool. I will start a pull-request.

Matt

Ok, I'll look for the pull request

I added a page in the wiki for the python version, it's just a copy of your readme for now but we should add some context at some point

also working on an update to the android subscriber so it can support more than one controller
I removed publisher mode because android would terminate the app when the device has no user interaction.

we'll have to add an extra field for the controller name in both the ESP and Python version

the MQTT topic is made up with
1 a root topic, with dioty, you have to use /youremail@mail.com/  , can be anything for mosquito as long as the root topic in the android settings and the publisher match
2 the controller name, so   /youremail@mail.com/MainController/ for dioty , in the app you would enter the controller name from the + when the nav bar is out (just like the classic's IP or URI)

I'll post the updated apk to google drive after a bit of testing

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 11, 2020, 02:11:06 PM
Awesome,

Can you give me the exact steps that you are using with "WAKE" and "INFO" - right now i am treating them the same, if I get either one, I reset the 5 minute countdown timer so that the every 5 seconds will continue.

Add the use of the "mac address" in the info screen <- It could also be used in the classic root, as it is unique to every Classic as is the unitid.

In the App, it appears that each page awaits the next message received every 5 seconds to populate, can you store all the data from each publish and then use that data to paint each screen as it is needed? That way when you flip from screen to screen, you do not have to wait for a message to come in to see the data.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 11, 2020, 02:44:38 PM
Quote from: mcsarge on March 11, 2020, 02:11:06 PM
Awesome,

Can you give me the exact steps that you are using with "WAKE" and "INFO" - right now i am treating them the same, if I get either one, I reset the 5 minute countdown timer so that the every 5 seconds will continue.

Add the use of the "mac address" in the info screen <- It could also be used in the classic root, as it is unique to every Classic as is the unitid.

In the App, it appears that each page awaits the next message received every 5 seconds to populate, can you store all the data from each publish and then use that data to paint each screen as it is needed? That way when you flip from screen to screen, you do not have to wait for a message to come in to see the data.

Matt

Info should make it publish the boilerplate info

topic: /graham.a.ross@gmail.com/Workshop/stat/info

payload:
{"appVersion":"","buildDate":"20180206","deviceName":"Workshop","deviceType":"Classic","endingAmps":10.81,"hasWhizbang":true,"lastVOC":13.21,"model":"Classic 200 (rev 4)","mpptMode":11,"netVersion":"","nominalBatteryVoltage":12,"unitID":-1901764051}


I'm just about ready to check in my code for the app and the esp32
in order to support more than one classic, I'm using the Unit Name at modbus address 4210 - 4213 (default to Classic if not set in the LA)
so in the above topic i have the / dioty root topic / Unit Name / stat / info

In the app, you would configure the MQTT broker, port, user, password and root topic from dioty for example /graham.a.ross@gmail.com/
then you would add an entry in the nav bar using the Unit Name


Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 11, 2020, 03:52:24 PM
OK,

I mis-spoke, when i get INFO or WAKE i transmit the boilerplate and the readings, and then send the readings every 5 seconds for 5 minutes. If I get a WAKE or INFO again, I transmit the boilerplate again and reset the countdown again.

Should we code the tools to automatically add the value we receive from that location on the Classic to the root? This will not be unique, however.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 12, 2020, 06:59:21 AM
Quote from: mcsarge on March 11, 2020, 03:52:24 PM
OK,

I mis-spoke, when i get INFO or WAKE i transmit the boilerplate and the readings, and then send the readings every 5 seconds for 5 minutes. If I get a WAKE or INFO again, I transmit the boilerplate again and reset the countdown again.

Should we code the tools to automatically add the value we receive from that location on the Classic to the root? This will not be unique, however.

Matt

The android app currently sends the wake every 58 seconds (just short of the 60 second publisher timer) to keep the readings coming every 2 seconds while the app is running, it doesn't need to get the info every 58 seconds since that data doesn't change.

Maybe it would be simpler just to add the controller name as an extra argument like --classicName <Workshop>

You can set the name at 4210 in the local app but maybe not everyone realizes that


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 12, 2020, 05:00:21 PM
I can change the code on the wake/info thing, but I was thinking that when running for long periods of time the boiler plate data might need to be updated, like last VOC and mpptMode. For the python code it makes no difference, I get it all every time no matter what, i just do not publish it if it is not time.

I think anyone with multiple Classics will know to set the names different. Just let me know how you want to proceed and I will update the python code.

I ordered a 10x5 inch (64x32 pixels) RGB LED matrix panel and HUB75 adapter to run it from an ESP32 - I am going to display the SOC, Voltage and some other data on it in the main cabin so that everyone will know if the genny needs to be started. I may add a defaulted parameter so I can set the snooze time from 5 minutes to 1 minute for my installation so the data on the panel does not get too old.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 12, 2020, 06:03:13 PM
I have my two Classics named different . Looking forward to the new mutliple Classic version. 

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 18, 2020, 01:19:06 PM
Quote from: ClassicCrazy on March 12, 2020, 06:03:13 PM
I have my two Classics named different . Looking forward to the new mutliple Classic version. 

Larry

I have an updated version (app-release-3.0.1.apk) on google drive https://drive.google.com/drive/folders/1S2BWLFn_nU0NmxXmyhJaz97h0K1_OY6B (https://drive.google.com/drive/folders/1S2BWLFn_nU0NmxXmyhJaz97h0K1_OY6B)

if you are using the Python implementation of the publisher, you can simply run two docker images, one for each controller.
get the latest from https://github.com/graham22/ClassicMQTT (https://github.com/graham22/ClassicMQTT) goto code/python and build the new docker image
docker build -t classic_mqtt .

here is an example (replace the classic <IP> mqtt_root, mqtt_user, mqtt_pass)

docker run -d classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name Cabin --mqtt mqtt.dioty.co --mqtt_root /graham.a.ross@gmail.com/ --mqtt_user graham.a.ross@gmail.com --mqtt_pass XXXXXXX
docker run -d classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name Workshop --mqtt mqtt.dioty.co --mqtt_root /graham.a.ross@gmail.com/ --mqtt_user graham.a.ross@gmail.com --mqtt_pass XXXXXX

Then setup the android app in MQTT mode with the MQTT broker, port, user, passwd, root topic
Add "Cabin" and "Workshop" to the nav bar

You should be able to switch back to MODBUS mode and set the classics IP as usual, hopefully I didn't break anything...

the pros;
can access the classics info from the internet without port forwarding if you setup a cloud based broker (dioty.co, Azure etc)
can use more than one app at the same time.
can be used with Home Assistant, NodeRed, or any other MQTT subscriber

cons
Day, Hour logs and Calendar view not available in MQTT mode



let me know how you do

Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 18, 2020, 02:35:32 PM
Thanks Graham - I will give this a try later on today or tomorrow.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 20, 2020, 01:00:33 AM
Graham ,
I followed  your instructions and  I have it all installed on Raspberry Pi using mosquitto and on my android phone .
Both my Classics show up and I can switch between the two and they both have data .
I like that the Messages info shows Aux - I used MNGP to turn AUX 1 on and it instantly showed up in the Messages info. 
It is night so not much happening with them now - but  I will look tomorrow to see the charging action.
I am surprised this all worked just the way it was supposed to first time I gave it a go - your instructions were good.
Thanks
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on March 20, 2020, 08:17:08 AM
Well, we did test it! :-) but I do not have 2 Classics so...

You might also notice that the MAC address is populated correctly and the Negative Amp Hours on "Capacity" is now displayed. It was not working on the MQTT version.

Matt

Quote from: ClassicCrazy on March 20, 2020, 01:00:33 AM
Graham ,
I followed  your instructions and  I have it all installed on Raspberry Pi using mosquitto and on my android phone .
Both my Classics show up and I can switch between the two and they both have data .
I like that the Messages info shows Aux - I used MNGP to turn AUX 1 on and it instantly showed up in the Messages info. 
It is night so not much happening with them now - but  I will look tomorrow to see the charging action.
I am surprised this all worked just the way it was supposed to first time I gave it a go - your instructions were good.
Thanks
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 20, 2020, 08:42:52 AM
Quote from: ClassicCrazy on March 20, 2020, 01:00:33 AM
Graham ,
I followed  your instructions and  I have it all installed on Raspberry Pi using mosquitto and on my android phone .
Both my Classics show up and I can switch between the two and they both have data .
I like that the Messages info shows Aux - I used MNGP to turn AUX 1 on and it instantly showed up in the Messages info. 
It is night so not much happening with them now - but  I will look tomorrow to see the charging action.
I am surprised this all worked just the way it was supposed to first time I gave it a go - your instructions were good.
Thanks
Larry

thanks for helping with the testing of the app
I've been running the publisher on both the ESP version and the Python implementation running in a docker image on linux both posting to the dioty.co broker for a few days without issue.

if both your classics feed one battery bank, you can now enable "system view" when in MQTT mode.
As you know, if you use a cloud based broker, the app can access the classic's data from anywhere on the internet when in MQTT mode,
you can also switch back to MODBUS mode when your device is on your local network to get the calendar information and to upload to PVOutput.

I have started a wiki on github for the project at https://github.com/graham22/ClassicMQTT/wiki (https://github.com/graham22/ClassicMQTT/wiki) that has public edit enabled, if you or anyone else would like to add more details, please go ahead.

I have version 3.0.2 on google drive https://drive.google.com/drive/folders/1S2BWLFn_nU0NmxXmyhJaz97h0K1_OY6B (https://drive.google.com/drive/folders/1S2BWLFn_nU0NmxXmyhJaz97h0K1_OY6B) with a few minor fixes.
I will not be posting this version of the app to google play since the setup might be beyond the scope of the average user.


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 20, 2020, 12:50:33 PM
Graham -
I just tried out System View - that is really nice feature and saves a lot of trouble flipping back and forth between controllers to see the total.
I may give the dioty server a try again - I did not have any luck with it when I first started on this.
My question about that is - do I have to change the settings in just the android app or do I have to redo the docker also ?
I set it up in docker just as you had said but I put in my mosquitto password - so should I run that again but with my dioty info ?
I am a bit confused about the difference between the Modbus and MQTT  settings in the Android app .

I just installed the latest Classic Monitor 3.02 and all is good here  - it kept all my previous settings.

Larry

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 20, 2020, 01:02:37 PM
Quote from: mcsarge on March 20, 2020, 08:17:08 AM
Well, we did test it! :-) but I do not have 2 Classics so...

You might also notice that the MAC address is populated correctly and the Negative Amp Hours on "Capacity" is now displayed. It was not working on the MQTT version.

Matt

Quote from: ClassicCrazy on March 20, 2020, 01:00:33 AM
Graham ,
I followed  your instructions and  I have it all installed on Raspberry Pi using mosquitto and on my android phone .
Both my Classics show up and I can switch between the two and they both have data .
I like that the Messages info shows Aux - I used MNGP to turn AUX 1 on and it instantly showed up in the Messages info. 
It is night so not much happening with them now - but  I will look tomorrow to see the charging action.
I am surprised this all worked just the way it was supposed to first time I gave it a go - your instructions were good.
Thanks
Larry

Thanks Matt - I noticed now that you mentioned it . This app has improved in a big way !
Your  collaboration with Graham  is much appreciated by me and I am sure everyone .

Larry

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on March 20, 2020, 01:27:02 PM
Quote from: ClassicCrazy on March 20, 2020, 12:50:33 PM
Graham -
I just tried out System View - that is really nice feature and saves a lot of trouble flipping back and forth between controllers to see the total.
I may give the dioty server a try again - I did not have any luck with it when I first started on this.
My question about that is - do I have to change the settings in just the android app or do I have to redo the docker also ?
I set it up in docker just as you had said but I put in my mosquitto password - so should I run that again but with my dioty info ?
I am a bit confused about the difference between the Modbus and MQTT  settings in the Android app .

I just installed the latest Classic Monitor 3.02 and all is good here  - it kept all my previous settings.

Larry

Yes both the Android app and the docker parameters must match.
You will need to stop your existing docker image with
docker ps 
docker stop <CONTAINER_ID>

(or just reboot the raspberry pi)

then run it with the dioty parameters you got from their confirmation email when you registered for the service.

here is an example of my parameters ;

docker run -d classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name Workshop --mqtt mqtt.dioty.co --mqtt_root /graham.a.ross@gmail.com/ --mqtt_user graham.a.ross@gmail.com --mqtt_pass XXXXXX

Set the same parameters in the android app settings for MQTT mode with the MQTT broker, User, Password and Root topic (including the slashes)
then add an entry in the nav bar using the name you have for the --classic_name parameter (Workshop in my case).

You can switch the app back to regular MODBUS mode where it will connect directly to your classic if you are on your local network as the current google play version 2.5.8 does.
The nav bar will switch to display the modbus connections you have instead of the MQTT names.

You will notice that when you hit the + on the nav bar in modbus mode, the dialog asks you for a port & IP/URI of the classic, when it's in MQTT mode, the add dialog just asks you for the name.

Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 20, 2020, 02:38:37 PM
I will give it a try later and see how it goes.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on March 31, 2020, 12:55:00 PM
Graham and Matt have worked out the bugs and the newest Beta Android Classic Monitoring  app is monitoring both of my Classics in follow me mode - the app can switch between getting the data directly from Modbus or you can switch it to monitor MQTT from either a Raspberry Pi or Esp32 ( though I think Esp32 only does one Classic but the Pi can monitor both at same time )
See the project  here for more details - new github address
https://github.com/ClassicDIY

Larry 
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Urayoan on October 03, 2020, 07:41:32 AM
Hello Graham:
  Setting up the ESP32 and men, how easy it was. Since I am using it, I noted the stage of the Classic never change from resting.

  One idea, is it possible to send to the ESP32 the wake command and then connect to the Classic if necessary? And if still connected keep the connection alive?

  I think with that approach, the modes side of the Classic is not blocked and let the LA run if the mobile app is closed


Thanks

PD. Just in case this helps, the JSON from the broker

{
   "BatTemperature":31.61,
   "NetAmpHours":0,
   "ChargeState":0,
   "InfoFlagsBits":-1308610556,
   "ReasonForResting":1,
   "NegativeAmpHours":0,
   "BatVoltage":54.11,
   "PVVoltage":141.01,
   "VbattRegSetPTmpComp":56.9,
   "TotalAmpHours":200,
   "WhizbangBatCurrent":0.01,
   "BatCurrent":9.81,
   "PVCurrent":3.91,
   "ConnectionState":0,
   "EnergyToday":0.21,
   "EqualizeTime":0,
   "SOC":100,
   "Aux1":false,
   "Aux2":false,
   "Power":533.01,
   "FETTemperature":46.91,
   "PositiveAmpHours":0,
   "TotalEnergy":1231.61,
   "FloatTimeTodaySeconds":5,
   "RemainingAmpHours":200,
   "AbsorbTime":7198,
   "ShuntTemperature":-49.99,
   "PCBTemperature":51.91
}
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on October 03, 2020, 10:54:23 AM
Quote from: Urayoan on October 03, 2020, 07:41:32 AM
Hello Graham:
  Setting up the ESP32 and men, how easy it was. Since I am using it, I noted the stage of the Classic never change from resting.

  One idea, is it possible to send to the ESP32 the wake command and then connect to the Classic if necessary? And if still connected keep the connection alive?

  I think with that approach, the modes side of the Classic is not blocked and let the LA run if the mobile app is closed


Thanks


Hi Urayoan

I fixed the issue with the charge state with the esp32 version of the ClassicMQTT publisher. see https://github.com/ClassicDIY/ClassicMQTT/releases/tag/1.3.3 (https://github.com/ClassicDIY/ClassicMQTT/releases/tag/1.3.3)

The modbus library I'm using for the ESP32 does not expose a disconnect method, I tried deleting the object but it wouldn't release the classic...

BTW: If you're interested I could add you as a dev on the ClassicDIY repo


Graham

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Urayoan on October 03, 2020, 11:27:33 AM
Thank you Graham, did the update and works like a charm!

Ill be honored if you add me to the contributors

Thank you again @Urayoan
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on October 08, 2020, 10:06:45 AM
Note, in the python version of the ClassicMQTT (run on a raspberry pi or other SBC), it connects, collects the modbus data and disconnects so you are able to sneak the LA in there between polling when you need to.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Urayoan on October 08, 2020, 10:16:00 AM
Thanks Matt. That means y need to acquire some kind of pi to experiment too!
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on October 08, 2020, 10:30:09 AM
It works on a Pi Zero - getting the Wifi version is the easiest. If you have trouble getting one, I have a ton I can send you one - for the cause!

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Urayoan on October 08, 2020, 10:41:44 AM
Quote from: mcsarge on October 08, 2020, 10:30:09 AM
It works on a Pi Zero - getting the Wifi version is the easiest. If you have trouble getting one, I have a ton I can send you one - for the cause!

Matt

Oh Thank you! Let me check first if i can find it and i let you know. When you mean the WiFi version, is because there is separate module to that? Like a hat (is that how is called?)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on October 08, 2020, 08:49:29 PM
Quote from: Urayoan on October 08, 2020, 10:41:44 AM
Quote from: mcsarge on October 08, 2020, 10:30:09 AM
It works on a Pi Zero - getting the Wifi version is the easiest. If you have trouble getting one, I have a ton I can send you one - for the cause!

Matt

Oh Thank you! Let me check first if i can find it and i let you know. When you mean the WiFi version, is because there is separate module to that? Like a hat (is that how is called?)

There is the Pi Zero that has no wifi and the Pi Zero W  which has the built in wifi.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Urayoan on October 08, 2020, 10:12:36 PM
Thank you Larry.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Frenergy on December 03, 2020, 12:27:18 PM
I seem to be cursed on this exercise.   

I tried first the ESP32 method and everything worked fine. Flashed the module, got success, reset the board and entered my SSID and PW - SAVE.   The port monitor showed the entries but then noting happened. I waited a long time and still nothing. I did the whole procedure 3 times. Always the same result. The Flasher does not invoke a browser window for me to complete the setup and I could never find the ESP32 device anywhere on my wi-fi network with a LAN scanner app, on the PC or tablet.   So ....

Being determined I figured I'd use the pi publisher version,  following the instructions provided.  Everything seemed to work fine until I entered the command (using my own proper variables of course):

"python3 classic_mqtt.py --classic <ClassicHost> --classic_port <502> ........................"

and got this response ....

Traceback (most recent call last):
File "classic_mqtt.py", line 3, in <module>
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
ModuleNotFoundError: No module named 'pymodbus'


I'm not hugely Linux savvy so maybe it's a PATH issue.  One problem I have when I install stuff is I have no idea where the program files have been put. 

Anyway, can anyone help please?  Providing precise Linux CLI commands to solve would be appreciated. Also, I need it to run every time I boot the Rpi3B+. 

TIA

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on December 03, 2020, 07:00:08 PM
Quote from: Frenergy on December 03, 2020, 12:27:18 PM
I seem to be cursed on this exercise.   

I tried first the ESP32 method and everything worked fine. Flashed the module, got success, reset the board and entered my SSID and PW - SAVE.   The port monitor showed the entries but then noting happened. I waited a long time and still nothing. I did the whole procedure 3 times. Always the same result. The Flasher does not invoke a browser window for me to complete the setup and I could never find the ESP32 device anywhere on my wi-fi network with a LAN scanner app, on the PC or tablet.   So ....

...

Anyway, can anyone help please?  Providing precise Linux CLI commands to solve would be appreciated. Also, I need it to run every time I boot the Rpi3B+. 

TIA

I can help with the ESP32 version.

If the blue LED is blinking on the ESP then it's in AP mode and needs to be configured to access your wifi. If the flasher program isn't able to send your wifi credentials to the esp you can configure it using a phone.

I made a short video on how to setup the ClassicMQTT using an android phone

https://youtu.be/ya6lDb4gbmk (https://youtu.be/ya6lDb4gbmk)

hope this helps
Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on December 17, 2020, 01:06:09 PM
There is alot to unpack there for the python version, I can help.

If you go with the docker solution, it will automatically load all the modules and stuff you need. If you just want to run it, you have to install that stuff yourself.

Have you read this Wiki page? It tells most of the story.

https://github.com/ClassicDIY/ClassicMQTT/wiki/3.-Raspberry-Pi-Classic-MQTT-Publisher

If I were you, I would go with instal #2 "Using Docker"

and make sure that you pay attention to the command that runs the image.
docker run classic_mqtt --classic <ClassicHost> --classic_port <502> --classic_name MyClassic --mqtt <127.0.0.1> --mqtt_port <1883> --mqtt_root <ClassicMQTT> --mqtt_user <username> --mqtt_pass <password>


If you are using Dioty, you need to run it in a similar fashion to this:

docker run classic_mqtt --classic 192.168.0.225 --classic_name MyClassic --mqtt mqtt.dioty.co --mqtt_root /joe.user@gmail.com/ClassicMQTT --mqtt_user joe.user@gmail.com --mqtt_pass <Joe's Dioty password>

If you want, send me the Dioty email message (with password overwritten) and I will build you the commands to run.

Matt

Matt


Quote from: Graham on December 03, 2020, 07:00:08 PM
Quote from: Frenergy on December 03, 2020, 12:27:18 PM
I seem to be cursed on this exercise.   

I tried first the ESP32 method and everything worked fine. Flashed the module, got success, reset the board and entered my SSID and PW - SAVE.   The port monitor showed the entries but then noting happened. I waited a long time and still nothing. I did the whole procedure 3 times. Always the same result. The Flasher does not invoke a browser window for me to complete the setup and I could never find the ESP32 device anywhere on my wi-fi network with a LAN scanner app, on the PC or tablet.   So ....

...

Anyway, can anyone help please?  Providing precise Linux CLI commands to solve would be appreciated. Also, I need it to run every time I boot the Rpi3B+. 

TIA

I can help with the ESP32 version.

If the blue LED is blinking on the ESP then it's in AP mode and needs to be configured to access your wifi. If the flasher program isn't able to send your wifi credentials to the esp you can configure it using a phone.

I made a short video on how to setup the ClassicMQTT using an android phone

https://youtu.be/ya6lDb4gbmk (https://youtu.be/ya6lDb4gbmk)

hope this helps
Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 11, 2021, 03:15:03 PM
I know this is a bit old but I am hoping someone can assist.  I was able to flash my ESP32, when I set up wifi by entering my ssid and password I get it shown on my cmd

{"ssid": "Homestead", "password": "mypassword"}

But the default browser does not open up and I do not see anything connected on my router.  Am I missing something?

Should I see anything else on the cmd interface?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 11, 2021, 05:30:03 PM
Quote from: ForeverHomestead on January 11, 2021, 03:15:03 PM
I know this is a bit old but I am hoping someone can assist.  I was able to flash my ESP32, when I set up wifi by entering my ssid and password I get it shown on my cmd

{"ssid": "Homestead", "password": "mypassword"}

But the default browser does not open up and I do not see anything connected on my router.  Am I missing something?

Should I see anything else on the cmd interface?
Are you able to get into the Lan list of your router and see if it is showing up there ?
Which IP address did you try connecting to ? 
You will have to connect to the one that is listed on your Lan .
If you see 192.168.4.1 try connecting to that .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 11, 2021, 05:36:19 PM
Quote from: ForeverHomestead on January 11, 2021, 03:15:03 PM
I know this is a bit old but I am hoping someone can assist.  I was able to flash my ESP32, when I set up wifi by entering my ssid and password I get it shown on my cmd

{"ssid": "Homestead", "password": "mypassword"}

But the default browser does not open up and I do not see anything connected on my router.  Am I missing something?

Should I see anything else on the cmd interface?

Also when I first tried flashing this I found out I needed to hold down the boot button on my ESP32 just before I flashed it - otherwise the flash wouldn't work.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 07:32:41 AM
I do not see it on the list.  It looks like the flash works, I get a prompt tp reboot, then I hit Wifi and it allows me to type it in and displays it on the screen, it just does not ever connect or give any indication that it is.

I tried holding the flash button down but it does not change the way it performs.

Is it supposed to have the blue light flashing showing it is in AP Mode?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 12, 2021, 11:11:20 AM
Quote from: ForeverHomestead on January 12, 2021, 07:32:41 AM
I do not see it on the list.  It looks like the flash works, I get a prompt tp reboot, then I hit Wifi and it allows me to type it in and displays it on the screen, it just does not ever connect or give any indication that it is.

I tried holding the flash button down but it does not change the way it performs.

Which Esp32 board are you using ?

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 12:07:39 PM
The board is the ESP32 Dev Kit.

Is there a certain board that works better than others?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 12, 2021, 01:03:37 PM
If you're having trouble setting the wifi credentials using the flasher tool you can set it using a phone by connecting to the ESP's access point

I made a youtube video showing the procedure

https://youtu.be/ya6lDb4gbmk (https://youtu.be/ya6lDb4gbmk)

hope this helps

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 01:17:38 PM
Hello Graham,

I watched your video yesterday.  I do not see a way to get the board into AP mode.  I do not see my board in my wifi connections on my phone.  Is there something special I need to do to get it into AP mode?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 12, 2021, 01:23:30 PM
Quote from: ForeverHomestead on January 12, 2021, 01:17:38 PM
Hello Graham,

I watched your video yesterday.  I do not see a way to get the board into AP mode.  I do not see my board in my wifi connections on my phone.  Is there something special I need to do to get it into AP mode?

Maybe it didn't get flashed properly .
Try flashing it again - don't bother with trying to put in the wifi credentials .
Then when done  you should see the LED flashing like Graham said .
Remember that just before you flash the Esp32 you have to hold in the Boot Button on it . I forget if you just need to hold this in when all the time it is flashing or just as you start flashing it.
The reason I asked about which board because there are probably different variations of them - some might not need the boot button held in when flashing it.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 12, 2021, 01:24:44 PM
Quote from: ForeverHomestead on January 12, 2021, 01:17:38 PM
Hello Graham,

I watched your video yesterday.  I do not see a way to get the board into AP mode.  I do not see my board in my wifi connections on my phone.  Is there something special I need to do to get it into AP mode?

If you hold GPIO15 to ground while powering the ESP up it will default to AP mode (blue led should flash)

if you do the same with GPIO4, this will clear all the data in nvram

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 02:53:18 PM
I get a pop up that says it is done.  The status bar on the flasher went to 100%  also the dos command screen shows

Has of data verified

Leaving.....
Hard Resetting via RTS pin


Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 02:55:05 PM
I see a pin labeled gnd and one labeled D15.  If I short those two it should go into ap mode?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 12, 2021, 03:03:37 PM
Quote from: ForeverHomestead on January 12, 2021, 02:55:05 PM
I see a pin labeled gnd and one labeled D15.  If I short those two it should go into ap mode?

yes, disconnect USB, connect D15 to ground, connect USB, should boot up in AP mode, remove jumper...
see attached picture
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 12, 2021, 05:12:04 PM
hmmmm.  That does not seem to do anything.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 13, 2021, 09:02:36 AM
Quote from: ForeverHomestead on January 12, 2021, 05:12:04 PM
hmmmm.  That does not seem to do anything.

It's likely your esp didn't get flashed correctly for some reason. I have had issues with USB cables, you might want to try another one...

I just tried it on one of my ESP dev boards that has ClassicMQTT flashed on it, with the GPIO15 (D15) connected to ground the blue led was flashing several times a second indicating it's in AP mode.
If there is data stored in the ESP for the Wifi credentials, the code will attempt to connect for 60 seconds, the led will blink slowly until it connects then will remain on if successful otherwise it will default to AP mode with the led flashing quickly.

You could also try flashing the ESP using the development IDE.
I have a video on how to flash the ESP with the VSCode development environment, I made it for another one of my projects but it's the same for ClassicMQTT , just get the code from https://github.com/ClassicDIY/ClassicMQTT (https://github.com/ClassicDIY/ClassicMQTT) instead of the one in the video. I made this video for someone in Poland so there is no audio.
https://youtu.be/GVixF3zlV1c (https://youtu.be/GVixF3zlV1c)
Once you build the project, use the Upload icon (->) to flash the esp, press and hold the 'boot' button on the esp for a few seconds to initiate the upload.

Once it connects to wifi (blue LED is on), I use Advanced IP Scanner https://www.advanced-ip-scanner.com/ (https://www.advanced-ip-scanner.com/) to find the ESP's ip address on the local network

Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 13, 2021, 01:39:37 PM
I was also going to suggest trying different usb cable and  different usb port .
Also maybe try a different esp32 example project that is supported on Arduino ( you have to add espboards to arduino ide  first though I believe) . That would be just to make sure the esp32 works.
And if on Windows use Device manager to make sure the Esp32 is showing up - this will also let you know for sure which port it is on. It won't say esp32 but whatever chip it uses for the serial port. I just unplug my esp32 from windows computer  while Device manager is open to see which one goes away .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 13, 2021, 04:28:07 PM
I am closer. lol

So I tried a new USB and port but that did not help. I downloaded the file and used Platformio to flash is and the blue light flashes.  I can see the classicmttp as an available connection, but unlike the video, when I click on it, it does not display setting, it asks for a password.  Whatever I put in says invalid.  Any ideas what the password would be?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 13, 2021, 04:46:21 PM
Oh so I used ClassicMQTT and it worked.  It did not open a page, but opeing a browser and going to 192.168.4.1 worked.

It does not like my wifi info though.  Strange because it works elsewhere.

Is there an issue connecting to a mesh network on this? I cannot choose 2.4 or 5 G, the Unifi router does it for me.  I think I had a problem with another device like this.

Oh, the plot thickens....

So I tried to use the other flasher again and entered the same wifi info on the flasher tool.  It seamed as though it did not work... then the blue LED went solid and a browser window opened with an ip address on it.  Nothing ever loaded and after about a minute the blue LED started flashing again....

Every once and awhilke it flashes faster, but then stops..
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 14, 2021, 07:34:12 AM
Quote from: ForeverHomestead on January 13, 2021, 04:46:21 PM
Oh so I used ClassicMQTT and it worked.  It did not open a page, but opeing a browser and going to 192.168.4.1 worked.

It does not like my wifi info though.  Strange because it works elsewhere.

Is there an issue connecting to a mesh network on this? I cannot choose 2.4 or 5 G, the Unifi router does it for me.  I think I had a problem with another device like this.

Oh, the plot thickens....

So I tried to use the other flasher again and entered the same wifi info on the flasher tool.  It seamed as though it did not work... then the blue LED went solid and a browser window opened with an ip address on it.  Nothing ever loaded and after about a minute the blue LED started flashing again....

Every once and awhilke it flashes faster, but then stops..

Sounds like it has a weak wifi signal, how far is it from your wifi router?

the default user/password is: admin/ClassicMQTT
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 14, 2021, 12:52:40 PM
Wifi router is not far at all.

I did notice something weird when I tried to flash it again.  There was a warning the crystal freq was detected at 40.01 MHz.  It is quite different than normalized freq of 40MHz.

I am off grid and have been equalizing our battery bank for the last couple days by running a generator.  Sometimes that will make a digital clock in my house lose time.  I wonder if there is an issue with this as well.  I will try this all again later tonight when I have the generator off.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 15, 2021, 06:59:50 AM
Quote from: ForeverHomestead on January 14, 2021, 12:52:40 PM
Wifi router is not far at all.

I did notice something weird when I tried to flash it again.  There was a warning the crystal freq was detected at 40.01 MHz.  It is quite different than normalized freq of 40MHz.

I am off grid and have been equalizing our battery bank for the last couple days by running a generator.  Sometimes that will make a digital clock in my house lose time.  I wonder if there is an issue with this as well.  I will try this all again later tonight when I have the generator off.

You could try flashing a simple test program to make sure you can establish a connection

https://circuits4you.com/2019/01/05/connecting-esp32-to-wifi-network/ (https://circuits4you.com/2019/01/05/connecting-esp32-to-wifi-network/)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 16, 2021, 07:54:58 AM
Interesting.  I never thought of connecting to my hotspot.  I put in credentials for my phones hotspot and it connected right away. 

I have had this before with another device...  a camera I think.  I am not sure why, but some things will not connect to my AmpliFi mesh network..

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 16, 2021, 09:35:28 AM
Quote from: ForeverHomestead on January 16, 2021, 07:54:58 AM
Interesting.  I never thought of connecting to my hotspot.  I put in credentials for my phones hotspot and it connected right away. 

I have had this before with another device...  a camera I think.  I am not sure why, but some things will not connect to my AmpliFi mesh network..

hmm... you could use the raspberry Pi implementation of ClassicMQTT, since the pi connects directly your ethernet, you won't have these wifi issues.

the other alternative to remote access is to use OpenVPN as described at IOTStack https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/ (https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/)

I use this approach now since it gives me remote access to both my home automation hub running HomeAssistant and my classic via my regular android app.

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 17, 2021, 10:47:29 AM
Quote from: Graham on January 16, 2021, 09:35:28 AM
Quote from: ForeverHomestead on January 16, 2021, 07:54:58 AM
Interesting.  I never thought of connecting to my hotspot.  I put in credentials for my phones hotspot and it connected right away. 

I have had this before with another device...  a camera I think.  I am not sure why, but some things will not connect to my AmpliFi mesh network..

hmm... you could use the raspberry Pi implementation of ClassicMQTT, since the pi connects directly your ethernet, you won't have these wifi issues.

the other alternative to remote access is to use OpenVPN as described at IOTStack https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/ (https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/)

I use this approach now since it gives me remote access to both my home automation hub running HomeAssistant and my classic via my regular android app.

Graham.

I wondered about the PI install as well.  I went with the ESP32 because it looked to be the simpler and less expensive way to go.  Looks like that may not work out.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 17, 2021, 12:52:31 PM
Quote from: ForeverHomestead on January 17, 2021, 10:47:29 AM
Quote from: Graham on January 16, 2021, 09:35:28 AM
Quote from: ForeverHomestead on January 16, 2021, 07:54:58 AM
Interesting.  I never thought of connecting to my hotspot.  I put in credentials for my phones hotspot and it connected right away. 

I have had this before with another device...  a camera I think.  I am not sure why, but some things will not connect to my AmpliFi mesh network..

hmm... you could use the raspberry Pi implementation of ClassicMQTT, since the pi connects directly your ethernet, you won't have these wifi issues.

the other alternative to remote access is to use OpenVPN as described at IOTStack https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/ (https://sensorsiot.github.io/IOTstack/Accessing-your-Device-from-the-internet/)

I use this approach now since it gives me remote access to both my home automation hub running HomeAssistant and my classic via my regular android app.

Graham.

I wondered about the PI install as well.  I went with the ESP32 because it looked to be the simpler and less expensive way to go.  Looks like that may not work out.

The Pi approach is better. There is a learning curve to all these microprocessor projects. Some of it can take a lot of time to figure out.  But Graham and Matt have documented everything about as well as anything I have seen on projects of this type.  The Pi runs Linux  and is more like a computer and can do different things compared to the Esp32 which just can do one thing when programmed.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 18, 2021, 06:57:37 AM
I ordered a pi 4, I will loop back around to this project when it comes in.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 18, 2021, 11:26:22 AM
Quote from: ForeverHomestead on January 18, 2021, 06:57:37 AM
I ordered a pi 4, I will loop back around to this project when it comes in.

Did you order the kit that comes with case, power supply , hdmi cable, sd card , etc ?
That is the best way to go.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 19, 2021, 07:01:38 AM
Yes, I ordered the starter kit.  I don't know anything about them, but I am techy so don't think it will be an issue.  I like the idea of using the pi as I can also add my own MQTT server to it.  Don't have to rely on internet access to get my data that way.

Thanks for the help guys. 
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 19, 2021, 11:13:37 AM
Quote from: ForeverHomestead on January 19, 2021, 07:01:38 AM
Yes, I ordered the starter kit.  I don't know anything about them, but I am techy so don't think it will be an issue.  I like the idea of using the pi as I can also add my own MQTT server to it.  Don't have to rely on internet access to get my data that way.

Thanks for the help guys.

After you get the Classic DIY Pi stuff going , check out this - it makes installing other good Pi programs easy. You won't have to install Mosquitto though since the Classic DIY already does that. I recommend installing Portainer since you can connect up to it on web browser and  it helps understand what programs are running under Docker.
Watch Andrew Spies video they reference
https://sensorsiot.github.io/IOTstack/Getting-Started/

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 19, 2021, 02:35:43 PM
Quote from: ClassicCrazy on January 19, 2021, 11:13:37 AM
...

After you get the Classic DIY Pi stuff going , check out this - it makes installing other good Pi programs easy. You won't have to install Mosquitto though since the Classic DIY already does that. I recommend installing Portainer since you can connect up to it on web browser and  it helps understand what programs are running under Docker.
Watch Andrew Spies video they reference
https://sensorsiot.github.io/IOTstack/Getting-Started/

Larry

I have some notes on one of my other ClassicDIY projects I'm tinkering with that use IOTStack to help setup the raspberry pi

https://github.com/ClassicDIY/BatteryTester/wiki/Raspberry-Pi-Setup (https://github.com/ClassicDIY/BatteryTester/wiki/Raspberry-Pi-Setup)

you will only need to select  Portainer-ce  Eclipse-Mosquito when you run IOTStack/menu.sh


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 19, 2021, 03:03:41 PM
Quote from: Graham on January 19, 2021, 02:35:43 PM
Quote from: ClassicCrazy on January 19, 2021, 11:13:37 AM
...

After you get the Classic DIY Pi stuff going , check out this - it makes installing other good Pi programs easy. You won't have to install Mosquitto though since the Classic DIY already does that. I recommend installing Portainer since you can connect up to it on web browser and  it helps understand what programs are running under Docker.
Watch Andrew Spies video they reference
https://sensorsiot.github.io/IOTstack/Getting-Started/

Larry
I have some notes on one of my other ClassicDIY projects I'm tinkering with that use IOTStack to help setup the raspberry pi

https://github.com/ClassicDIY/BatteryTester/wiki/Raspberry-Pi-Setup (https://github.com/ClassicDIY/BatteryTester/wiki/Raspberry-Pi-Setup)

you will only need to select  Portainer-ce  Eclipse-Mosquito when you run IOTStack/menu.sh


Graham.

The other day I was trying to figure out why the Classic pi mqtt worked with the mosquitto originally installed but wouldn't work with the Mosquito that I installed with IOT Stack. I used Portainer and  turned off the one that was in original docker ( under Python container ) and then copied some of the mosquitto files from that version over to the mosquitto installed by IOTstack . It didn't work and I gave up on it - didn't want to put any more time into that right then.  Not sure if it matters - I was just curious .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 19, 2021, 03:08:35 PM
Graham -
Not sure but you might want to note in your instructions that sudo apt full-upgrade only works on the new version of Pi Buster - it won't work on the older version. In case someone is already using a pi from years back with old Pi OS.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 20, 2021, 08:59:47 AM
Quote from: ClassicCrazy on January 19, 2021, 03:08:35 PM
Graham -
Not sure but you might want to note in your instructions that sudo apt full-upgrade only works on the new version of Pi Buster - it won't work on the older version. In case someone is already using a pi from years back with old Pi OS.

Larry

ok thanks Larry, added the note...

Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 22, 2021, 05:05:46 PM
Hey Guys,

So my pi came today and I am working on setting things up.  I am a complete newbie when it comes to Pi.  I am confused about the step that says:

QuoteCreate the .env file and specify the 5 items listed below in this format. They correspond to the parameters for the classic_mqtt.py program. Notice that since we are bringing up our own MQTT Broker, we can skip specifying the MQTT host. The last 3 parameters will work with the included MQTT broker, so there is no need to change those. To create this file on the Raspberry Pi, I like nano.
CLASSIC=<IP address or URL> 
CLASSIC_PORT=<Port usually 502>
CLASSIC_NAME=MyClassic
MQTT_ROOT=ClassicMQTT
MQTT_USER=ClassicPublisher
MQTT_PASS=ClassicPub123

I am not sure what/how I create this file or where it should be saved.  What am I missing
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 22, 2021, 09:54:33 PM
Quote from: ForeverHomestead on January 22, 2021, 05:05:46 PM
Hey Guys,

So my pi came today and I am working on setting things up.  I am a complete newbie when it comes to Pi.  I am confused about the step that says:

QuoteCreate the .env file and specify the 5 items listed below in this format. They correspond to the parameters for the classic_mqtt.py program. Notice that since we are bringing up our own MQTT Broker, we can skip specifying the MQTT host. The last 3 parameters will work with the included MQTT broker, so there is no need to change those. To create this file on the Raspberry Pi, I like nano.
CLASSIC=<IP address or URL> 
CLASSIC_PORT=<Port usually 502>
CLASSIC_NAME=MyClassic
MQTT_ROOT=ClassicMQTT
MQTT_USER=ClassicPublisher
MQTT_PASS=ClassicPub123

I am not sure what/how I create this file or where it should be saved.  What am I missing

go to your folder where Classic MQTT-Master files are
Go to the Code folder
Go to Python folder
then do
sudo nano .env
It should open up a file with nothing in it - copy and past the stuff above in it with your changes for your system
then do
Control X
yes to save changes
yes to save in the file name
If you want to check it after it closes you can do
sudo nano .env 
do control x to close it and it will just close if you don't make any changes
The .env might not show up in the files after you make it but you can tell if it is there if you do the nano .env
I told you how to do this using terminal program - but you can probably do same thing using desktop file manager

Larry



Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 23, 2021, 07:18:54 AM
that is perfect.  I did some searching and was only missing a step.  Thanks.

If I have multiple Classics, what is the process of adding them into this as well?  Do I just add more lines to the file with the ip address and names?

Looks like I am getting an error trying to read the data from the classic:

classic_mqtt_1  | 2021-01-23 13:37:41,819:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-23 13:37:47,830:ERROR:classic_mqtt:MODBUS Error H:192.168.2.238 P:502 e:<class 'pymodbus.exceptions.ConnectionException'>
classic_mqtt_1  | 2021-01-23 13:37:47,831:ERROR:classic_mqtt:MODBUS data not good, skipping publish
classic_mqtt_1  | 2021-01-23 13:37:47,831:DEBUG:classic_mqtt:Adjusting next interval to account for cycle taking too long: -1.0118117332458496
classic_mqtt_1  | 2021-01-23 13:37:47,832:DEBUG:classic_mqtt:Adjusted interval: 3.9881882667541504




Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 23, 2021, 09:20:36 AM
Quote from: ForeverHomestead on January 23, 2021, 07:18:54 AM
that is perfect.  I did some searching and was only missing a step.  Thanks.

If I have multiple Classics, what is the process of adding them into this as well?  Do I just add more lines to the file with the ip address and names?

Looks like I am getting an error trying to read the data from the classic:

classic_mqtt_1  | 2021-01-23 13:37:41,819:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-23 13:37:47,830:ERROR:classic_mqtt:MODBUS Error H:192.168.2.238 P:502 e:<class 'pymodbus.exceptions.ConnectionException'>
classic_mqtt_1  | 2021-01-23 13:37:47,831:ERROR:classic_mqtt:MODBUS data not good, skipping publish
classic_mqtt_1  | 2021-01-23 13:37:47,831:DEBUG:classic_mqtt:Adjusting next interval to account for cycle taking too long: -1.0118117332458496
classic_mqtt_1  | 2021-01-23 13:37:47,832:DEBUG:classic_mqtt:Adjusted interval: 3.9881882667541504

Looking at the python code doesn't give any further detail... (see attachment)
try pinging your classic from the pi
ping 192.168.2.238
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 23, 2021, 11:16:20 AM
ping is fine.  I dont need to change anything on the classic itself do I?  Not much to set in there really.  I am set as DHCP, am I better to set as static?  Right now I have my router set to always give it the same address.

I dont see any error in the startup.

/usr/local/lib/python3.5/dist-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.5 support will be dropped in the next release of cryptography. Please upgrade your Python.
  from cryptography.hazmat.backends import default_backend
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Starting python_mosquitto_1 ... done
Starting python_classic_mqtt_1 ... done
Attaching to python_mosquitto_1, python_classic_mqtt_1
mosquitto_1     | chown: /mosquitto/config/mosquitto.conf: Read-only file system
mosquitto_1     | 1611418640: mosquitto version 1.6.12 starting
classic_mqtt_1  | 2021-01-23 16:17:23,690:INFO:classic_mqtt:classic_mqtt starting up...
classic_mqtt_1  | 2021-01-23 16:17:23,706:INFO:classic_mqtt:classicHost = 192.168.2.238
classic_mqtt_1  | 2021-01-23 16:17:23,707:INFO:classic_mqtt:classicPort = 502
classic_mqtt_1  | 2021-01-23 16:17:23,707:INFO:classic_mqtt:classicName = Classic_1
classic_mqtt_1  | 2021-01-23 16:17:23,708:INFO:classic_mqtt:mqttHost = mosquitto
classic_mqtt_1  | 2021-01-23 16:17:23,709:INFO:classic_mqtt:mqttPort = 1883
classic_mqtt_1  | 2021-01-23 16:17:23,709:INFO:classic_mqtt:mqttRoot = ClassicMQTT
classic_mqtt_1  | 2021-01-23 16:17:23,709:INFO:classic_mqtt:mqttUser = ClassicPublisher
classic_mqtt_1  | 2021-01-23 16:17:23,710:INFO:classic_mqtt:mqttPassword = **********
classic_mqtt_1  | 2021-01-23 16:17:23,711:INFO:classic_mqtt:awakePublishRate = 5
classic_mqtt_1  | 2021-01-23 16:17:23,711:INFO:classic_mqtt:snoozePublishRate = 300
classic_mqtt_1  | 2021-01-23 16:17:23,711:INFO:classic_mqtt:awakePublishLimit = 60
classic_mqtt_1  | 2021-01-23 16:17:23,712:DEBUG:classic_mqtt:snoozeCycleLimit: 60
classic_mqtt_1  | 2021-01-23 16:17:23,712:INFO:classic_mqtt:Connecting with clientId=ClassicPublisher_mqttclient_245
classic_mqtt_1  | 2021-01-23 16:17:23,714:INFO:classic_mqtt:Connecting to MQTT mosquitto:1883
classic_mqtt_1  | 2021-01-23 16:17:23,750:DEBUG:classic_mqtt:MQTT connected OK Returned code=0
classic_mqtt_1  | 2021-01-23 16:17:23,752:DEBUG:classic_mqtt:Starting main loop...
classic_mqtt_1  | 2021-01-23 16:17:23,754:DEBUG:classic_mqtt:Subscribed to ClassicMQTT/Classic_1/cmnd/#
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 23, 2021, 11:31:59 AM
Quote from: ForeverHomestead on January 23, 2021, 11:16:20 AM
ping is fine.  I dont need to change anything on the classic itself do I?  Not much to set in there really.  I am set as DHCP, am I better to set as static?  Right now I have my router set to always give it the same address.
...

can you access your classic from a pc running the midnite local app?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 12:01:29 PM
Quote from: ForeverHomestead on January 23, 2021, 07:18:54 AM
that is perfect.  I did some searching and was only missing a step.  Thanks.

If I have multiple Classics, what is the process of adding them into this as well?  Do I just add more lines to the file with the ip address and names?

Looks like I am getting an error trying to read the data from the classic:

classic_mqtt_1  | 2021-01-23 13:37:41,819:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-23 13:37:47,830:ERROR:classic_mqtt:MODBUS Error H:192.168.2.238 P:502 e:<class 'pymodbus.exceptions.ConnectionException'>
classic_mqtt_1  | 2021-01-23 13:37:47,831:ERROR:classic_mqtt:MODBUS data not good, skipping publish
classic_mqtt_1  | 2021-01-23 13:37:47,831:DEBUG:classic_mqtt:Adjusting next interval to account for cycle taking too long: -1.0118117332458496
classic_mqtt_1  | 2021-01-23 13:37:47,832:DEBUG:classic_mqtt:Adjusted interval: 3.9881882667541504

The .env file I showed in screen shot for mine is how you set it up for two Classics
But you also need to make the second python file like you did for the first one according to the directions ( but put in the info for second Classic )

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 23, 2021, 12:02:59 PM
Yes I can access it with the local app.

I think it may just be an occasional error.  I think maybe it is dropping the connection.  I may move it out into my power house and connect via lan in the same switch as the classics.

Can I view the data that it collects via a data stream or folder?  I see the log talking about that, I do not see a folder on the pie though. 
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 12:03:18 PM
Getting Portainer running is a big help to see if the docker files are running .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 12:13:26 PM
This is what portainer looks like .
As you can see I have two versions of Mosquitto installed by Docker.
One is not running.
Using  IOT Stack to install Portainer CE is the easiest way .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 23, 2021, 12:19:59 PM
You could use MQTT.fx and subscribe to the root topic ClassicMQTT/# to see if the python code is publishing the classics data to your broker

https://mqttfx.jensd.de/index.php/download (https://mqttfx.jensd.de/index.php/download)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 23, 2021, 04:00:53 PM
I see you have a CLASSIC_NAME1, CLASSIC_NAME2 but no CLASSIC_NAME.... without a number.  I set mine like that and is complains about not having the one without CLASSIC_NAME.  Did you have that issue as well?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 08:40:12 PM
Quote from: ForeverHomestead on January 23, 2021, 04:00:53 PM
I see you have a CLASSIC_NAME1, CLASSIC_NAME2 but no CLASSIC_NAME.... without a number.  I set mine like that and is complains about not having the one without CLASSIC_NAME.  Did you have that issue as well?

You should set yours up with just one Classic at first to make sure you have everything working.
Follow the instructions. Use the .env file in the instructions for just one Classic.
If you get it working on just one Classic then you can try to get both of them working.
I think some of the names might be case sensitive so make sure .env file matches the other places you might put in Classic names.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 08:49:10 PM
This is my classic-compose yaml file .
Mat helped me set this all up to work with two classics .
That is why I thought  you should try to just get one classic working as is documented .
It could be that is why you are getting errors because one of your other files isn't supporting two Classics
I probably shouldn't  have posted my .env file since it is sort of customized.
Anyway this is the yaml file that goes with it which may have been modified too

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 23, 2021, 11:03:19 PM
Graham/Larry/Matt,   Thanks for setting up this Classic to MQTT tool, and for the great packaged setup.  It works!

Installed on a Pi with crossover cable to the Classic, and the Pi is set up as bridge/router from home wifi.  The Pi is set up using IOTstack with MQTT to NodeRed to influxdb to grafana.  The Pi collects data from 2 Enphase Envoys, a bridge to the utility main meter, a pair of SMA inverters, and now the Classic!  Woohoo!

I've been trying to figure out the SMA CAN messages enough to be useful for controlling (haven't found battery voltage & current signals correctly yet).  It's been on my list to connect up to the Classic, but it dawned on me that it has battery voltage.   It didn't take long to plug in and get it working.   
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 23, 2021, 11:43:39 PM
Quote from: daklein on January 23, 2021, 11:03:19 PM
Graham/Larry/Matt,   Thanks for setting up this Classic to MQTT tool, and for the great packaged setup.  It works!

Installed on a Pi with crossover cable to the Classic, and the Pi is set up as bridge/router from home wifi.  The Pi is set up using IOTstack with MQTT to NodeRed to influxdb to grafana.  The Pi collects data from 2 Enphase Envoys, a bridge to the utility main meter, a pair of SMA inverters, and now the Classic!  Woohoo!

I've been trying to figure out the SMA CAN messages enough to be useful for controlling (haven't found battery voltage & current signals correctly yet).  It's been on my list to connect up to the Classic, but it dawned on me that it has battery voltage.   It didn't take long to plug in and get it working.   

Glad you got it all working . Could you post a screen shot of the results you get on grafana ? I have been wanting to do that so any details you can share would be appreciated.  Since you are taking the data to Node Red could you share your flow and maybe that will show enough of how to make that setup work?   I also never thought about setting up the pi as a router .  Thanks

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 08:35:58 AM
Looks like it is working with one classic:

classic_mqtt_1  | 2021-01-24 08:18:46,794:DEBUG:classic_mqtt:Got data from Classic at 192.168.2.238:502
classic_mqtt_1  | 2021-01-24 08:18:46,804:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/readings
classic_mqtt_1  | 2021-01-24 08:18:46,807:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/info
classic_mqtt_1  | 2021-01-24 08:23:51,797:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-24 08:23:51,912:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
classic_mqtt_1  | 2021-01-24 08:23:51,913:DEBUG:classic_mqtt:Got data from Classic at 192.168.2.238:502
classic_mqtt_1  | 2021-01-24 08:23:51,916:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/readings
classic_mqtt_1  | 2021-01-24 08:23:51,917:DEBUG:classic_mqtt:Publishing: ClassicMQTT/Classic_1/stat/info
classic_mqtt_1  | 2021-01-24 08:28:56,874:DEBUG:classic_mqtt:Opening the modbus Connection
classic_mqtt_1  | 2021-01-24 08:28:57,133:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode

I will work on getting the other items installed on the pi
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 24, 2021, 09:25:59 AM
You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 09:37:39 AM
Quote from: Graham on January 24, 2021, 09:25:59 AM
You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82

That is perfect thanks for the info.  I will work on that once I know that the first is working.  I have yet to get the info that it is publishing yet.  The app is not displaying any info.

I just got Portainer installed.  Still working on fuguring out how to view what was published.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 10:24:10 AM
Ok, I have it working with the first of 3 Classics. 

How do I delete a device from the app?  I entered the wrong name the first time.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 24, 2021, 10:31:07 AM
Quote from: ForeverHomestead on January 24, 2021, 10:24:10 AM
Ok, I have it working with the first of 3 Classics. 

How do I delete a device from the app?  I entered the wrong name the first time.

just slide it to the right
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 10:57:10 AM
Quote from: Graham on January 24, 2021, 10:31:07 AM

just slide it to the right

Well wasnt that simple.  lol
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 12:06:31 PM
Quote from: Graham on January 24, 2021, 09:25:59 AM
You will need two docker instances running on the pi, each instance configured to each classic

here is an example

docker run -d --name=classicMQTT1 classic_mqtt --classic 192.168.86.37 --classic_port 502 --classic_name CLASSIC2  --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

docker run -d --name=classicMQTT2 classic_mqtt --classic 192.168.86.32 --classic_port 502 --classic_name HoopPV --mqtt 192.168.86.82 --mqtt_root ClassicMQTT --mqtt_user HASS --mqtt_pass myPasswd

the first docker instance "ClassicMQTT1" reads data from my first classic 192.168.86.37 and publishes MQTT to my local broker at 192.168.86.82
the second docker instance "ClassicMQTT2" reads data from my second classic 192.168.86.32 and publishes MQTT to the same local broker at 192.168.86.82

So if I run the file l;ike this. (Looks like example 2 in the how to)  Do I still need to make the .env file and run the build and up like in the 3rd example?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 24, 2021, 12:18:13 PM
If you do the two Docker instances like Graham said then make your .env file like I had in my screenshot . Just make sure that what you name your Classics matches what you have in the .env  ( case sensitive )

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 24, 2021, 12:40:16 PM
Quote from: ForeverHomestead on January 24, 2021, 12:06:31 PM
...

So if I run the file l;ike this. (Looks like example 2 in the how to)  Do I still need to make the .env file and run the build and up like in the 3rd example?

no, i"m passing in the arguments as parameters as in "2 Using docker" from the wiki so you don't need the env file

I'm assuming you already have the docker image built for ClassicMQTT , if not just run

docker build -t classic_mqtt .

also assuming you have the mosquito broker image running, use portainer to see what you have running...

if you want to use the "3. Using docker-compose" method then you would need to change the .yml like Larry has it in his screenshot
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 24, 2021, 01:55:15 PM
That is perfect.  So the last remaining step for me is a way to get this data in to my home automation server.  I use a program called Homeseer.  To get the information from my Mate 3 I was able to write a script to parse the Json output of the mate.  What are my options for getting the information sent to Mosqitto?  is there a viewable output via web or somehng similar?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 24, 2021, 02:29:48 PM
Quote from: ForeverHomestead on January 24, 2021, 01:55:15 PM
That is perfect.  So the last remaining step for me is a way to get this data in to my home automation server.  I use a program called Homeseer.  To get the information from my Mate 3 I was able to write a script to parse the Json output of the mate.  What are my options for getting the information sent to Mosqitto?  is there a viewable output via web or somehng similar?

You could use NodeRed to subscribe and publish to your mosquito broker

https://cookbook.nodered.org/#mqtt (https://cookbook.nodered.org/#mqtt)

I see node red has a homeseer plugin

https://flows.nodered.org/node/node-red-contrib-homeseer (https://flows.nodered.org/node/node-red-contrib-homeseer)

if you just want to view data in the broker you can use http://mqtt-explorer.com/ (http://mqtt-explorer.com/) or https://mqttfx.jensd.de/ (https://mqttfx.jensd.de/)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 25, 2021, 04:53:01 PM
everything working on first Classic.  Tried adding the other two based on the image shown above and I get an error.  Any ideas what is wrong>

    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue.
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #

    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${M$
classic_mqtt2:
   image: classic_mqtt
   restart: always
   build: .
   environment:
     - LOGLEVEL=DEBUG
   depends_on:
     - mosquitto
   networks:
     - localnet
   command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass $
classic_mqtt3:
   image: classic_mqtt
   restart: always
   build: .
   environment:
     - LOGLEVEL=DEBUG
   depends_on:
     - mosquitto
   networks:
     - localnet
   command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass $
networks:
  localnet:


error is:

ERROR: yaml.parser.ParserError: while parsing a block mapping
  in "./classic_mqtt_compose.yml", line 1, column 1
expected <block end>, but found '<block mapping start>'
  in "./classic_mqtt_compose.yml", line 50, column 2
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 25, 2021, 07:37:31 PM
So I had some spacing issues in the above.  I ran it through a validator and got that fixed, now the error has changed to:

ERROR: The Compose file './classic_mqtt_compose.yml' is invalid because:
'classic_mqtt2', 'classic_mqtt3' do not match any of the regexes: '^x-'

You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/


Any ideas?
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 25, 2021, 11:59:38 PM
Quote from: ClassicCrazy on January 23, 2021, 11:43:39 PM
Quote from: daklein on January 23, 2021, 11:03:19 PM
Graham/Larry/Matt,   Thanks for setting up this Classic to MQTT tool, and for the great packaged setup.  It works!

Installed on a Pi with crossover cable to the Classic, and the Pi is set up as bridge/router from home wifi.  The Pi is set up using IOTstack with MQTT to NodeRed to influxdb to grafana.  The Pi collects data from 2 Enphase Envoys, a bridge to the utility main meter, a pair of SMA inverters, and now the Classic!  Woohoo!

I've been trying to figure out the SMA CAN messages enough to be useful for controlling (haven't found battery voltage & current signals correctly yet).  It's been on my list to connect up to the Classic, but it dawned on me that it has battery voltage.   It didn't take long to plug in and get it working.   

Glad you got it all working . Could you post a screen shot of the results you get on grafana ? I have been wanting to do that so any details you can share would be appreciated.  Since you are taking the data to Node Red could you share your flow and maybe that will show enough of how to make that setup work?   I also never thought about setting up the pi as a router .  Thanks

Larry

Tonight I got it working in the IOTstack system, running from docker-compose, with some help from the IOTstack discord.  Next it may get included as a selection in IOTstack.   

For now, I added just battery voltage measured from the Midnite, to the plots that mostly have data from the SMA inverters with fewer signals than I want properly scaled. Battery voltage will be very helpful, plus I'll add in the Midnite solar production to the plots.   

Here's a plot attached, Great fun!!     
      Little red line in upper plot, MN_BatV
      Upper plot timescale is zoomed, lower two plots are wider timescale
      The spikes are the SMA inverters going in & out of 'Silent' mode, charging for a short time w/ battery pretty full alreadt, then letting it sit with inverters off, while connected to the grid.

Here's the nodered flow I used to go from the mqtt message to influxdb:   
[{"id":"a11c0bc7.6343a8","type":"mqtt in","z":"502d1323.9f228c","name":"stat/readings","topic":"ClassicMQTT/MNC250CP/stat/readings","qos":"0","datatype":"json","broker":"724ca09d.3fbec","x":150,"y":140,"wires":[["6d87693e.9e2c98","835d8d89.09859"]]},{"id":"6d87693e.9e2c98","type":"debug","z":"502d1323.9f228c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":350,"y":180,"wires":[]},{"id":"944e24d5.0151c8","type":"influxdb out","z":"502d1323.9f228c","influxdb":"98b7f44b.9b4d48","name":"influx","measurement":"MN_BatV","precision":"","retentionPolicy":"","database":"Energy","precisionV18FluxV20":"s","retentionPolicyV18Flux":"","org":"organisation","bucket":"bucket","x":670,"y":140,"wires":[]},{"id":"835d8d89.09859","type":"change","z":"502d1323.9f228c","name":"MN_BatV","rules":[{"t":"set","p":"payload","pt":"msg","to":"{   \"MN_BatV\":payload.BatVoltage}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":140,"wires":[["d803ad69.c1a8f","944e24d5.0151c8"]]},{"id":"d803ad69.c1a8f","type":"debug","z":"502d1323.9f228c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":630,"y":180,"wires":[]},{"id":"724ca09d.3fbec","type":"mqtt-broker","name":"","broker":"192.168.15.15","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"98b7f44b.9b4d48","type":"influxdb","hostname":"127.0.0.1","port":"8086","protocol":"http","database":"EnergyBridgeDB","name":"","usetls":false,"tls":"","influxdbVersion":"1.8-flux","url":"http://influxdb:8086","rejectUnauthorized":true}]
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 26, 2021, 02:11:45 AM
Thanks daklein
I will give the node red flow  a try next time I get a chance .
Your other post about how to make Iot container  from github is interesting too.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 26, 2021, 08:14:50 AM
Quote from: ForeverHomestead on January 25, 2021, 07:37:31 PM
So I had some spacing issues in the above.  I ran it through a validator and got that fixed, now the error has changed to:

ERROR: The Compose file './classic_mqtt_compose.yml' is invalid because:
'classic_mqtt2', 'classic_mqtt3' do not match any of the regexes: '^x-'

You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/


Any ideas?

is that your whole .yml file in your previous post? or do you have the following at the top before the comment block?


version: "3"
services:
  classic_mqtt:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 26, 2021, 08:43:48 AM
Yes that is at the top.  Looks like it did not copy the whole thing over.  Here is the complete:

version: "3"
services:
  mosquitto:
    hostname: mosquitto
    image: eclipse-mosquitto:latest
    restart: always
    volumes:
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - ./mosquitto/mosquitto.passwd:/mosquitto/config/mosquitto.passwd
    networks:
      - localnet
    ports:
      - "1883:1883"
  classic_mqtt:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet

    # Note you need to create your own .env file in the same directory as this yml file.
    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue. 
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #
    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
classic_mqtt2:
  image: classic_mqtt
  restart: always
  build: .
  environment:
    - LOGLEVEL=DEBUG
  depends_on:
    - mosquitto
  networks:
    - localnet
  command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
classic_mqtt3:
  image: classic_mqtt
  restart: always
  build: .
  environment:
    - LOGLEVEL=DEBUG
  depends_on:
    - mosquitto
  networks:
    - localnet
  command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
networks:
  localnet:
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 26, 2021, 09:15:35 AM
Quote from: ForeverHomestead on January 26, 2021, 08:43:48 AM
Yes that is at the top.  Looks like it did not copy the whole thing over.  Here is the complete:

version: "3"
services:
  mosquitto:
    hostname: mosquitto
    image: eclipse-mosquitto:latest
    restart: always
    volumes:
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - ./mosquitto/mosquitto.passwd:/mosquitto/config/mosquitto.passwd
    networks:
      - localnet
    ports:
      - "1883:1883"
  classic_mqtt:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet

    # Note you need to create your own .env file in the same directory as this yml file.
    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue. 
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #
    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
classic_mqtt2:
  image: classic_mqtt
  restart: always
  build: .
  environment:
    - LOGLEVEL=DEBUG
  depends_on:
    - mosquitto
  networks:
    - localnet
  command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
classic_mqtt3:
  image: classic_mqtt
  restart: always
  build: .
  environment:
    - LOGLEVEL=DEBUG
  depends_on:
    - mosquitto
  networks:
    - localnet
  command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
networks:
  localnet:


you need to add 2 spaces before each line starting at classic_mqtt2: to the last command:
the indentation has to match the first classic_mqtt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 26, 2021, 09:45:30 AM
hmmm.  Replaced with this error:

ERROR: The Compose file './classic_mqtt_compose.yml' is invalid because:
Unsupported config option for services.networks: 'localnet'

version: "3"
services:
  mosquitto:
    hostname: mosquitto
    image: eclipse-mosquitto:latest
    restart: always
    volumes:
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - ./mosquitto/mosquitto.passwd:/mosquitto/config/mosquitto.passwd
    networks:
      - localnet
    ports:
      - "1883:1883"
  classic_mqtt:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet

    # Note you need to create your own .env file in the same directory as this yml file.
    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue. 
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #
    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  classic_mqtt2:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
    command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  classic_mqtt3:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
    command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  networks:
    localnet:

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 26, 2021, 09:50:50 AM
Quote from: ForeverHomestead on January 26, 2021, 09:45:30 AM
hmmm.  Replaced with this error:

ERROR: The Compose file './classic_mqtt_compose.yml' is invalid because:
Unsupported config option for services.networks: 'localnet'

version: "3"
services:
  mosquitto:
    hostname: mosquitto
    image: eclipse-mosquitto:latest
    restart: always
    volumes:
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - ./mosquitto/mosquitto.passwd:/mosquitto/config/mosquitto.passwd
    networks:
      - localnet
    ports:
      - "1883:1883"
  classic_mqtt:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet

    # Note you need to create your own .env file in the same directory as this yml file.
    # In it add ALL of the ${VARIABLES} used below. In windows, VS Code lets you create ".env" easily,
    # but in file explorer you cannot. On Linux, no issue. 
    # Note, in the mosquitto.passwd file in the repo there are 2 users or you can create your own:
    # ClassicPublisher password = ClassicPub123
    # ClassicClient password= ClassicClient123
    #
    # The file should look like the following.
    #
    # CLASSIC=<IP address or URL>
    # CLASSIC_PORT=<Port usually 502>
    # CLASSIC_NAME=<Name of you Classic usually "classic" - the name used in Android App>
    # MQTT_ROOT=<The MQTT Root, usually ClassicMQTT>
    # MQTT_USER=ClassicPublisher
    # MQTT_PASS=ClassicPub123
    #
    # After you have created the .env file issue the following command to build and run it:
    # docker-compose -f classic_mqtt_compose.yml up
    #
    # Note, if you change the .env file after you run this command the first time, you need to issue
    # the following command to re-build it before running it again.
    # docker-compose -f classic_mqtt_compose.yml build
    #
    command: "--classic_name ${CLASSIC_NAME} --classic ${CLASSIC} --classic_port ${CLASSIC_PORT} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  classic_mqtt2:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
    command: "--classic_name ${CLASSIC_NAME2}  --classic ${CLASSIC2} --classic_port ${CLASSIC_PORT2} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  classic_mqtt3:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
    command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
  networks:
    localnet:


don't add spaces to the last two lines, just to the command: ...


  classic_mqtt3:
    image: classic_mqtt
    restart: always
    build: .
    environment:
      - LOGLEVEL=DEBUG
    depends_on:
      - mosquitto
    networks:
      - localnet
    command: "--classic_name ${CLASSIC_NAME3}  --classic ${CLASSIC3} --classic_port ${CLASSIC_PORT3} --mqtt mosquitto --mqtt_root ${MQTT_ROOT} --mqtt_user ${MQTT_USER} --mqtt_pass ${MQTT_PASS}"
networks:
  localnet:
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 26, 2021, 09:54:16 AM
Oh shoot, ok I was going to ask that.  That did it.  Looks like it is building now.  Thanks for the help through this. 
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 26, 2021, 10:32:11 AM
Quote from: ForeverHomestead on January 26, 2021, 09:54:16 AM
Oh shoot, ok I was going to ask that.  That did it.  Looks like it is building now.  Thanks for the help through this.

Just tested it with three classics (one real one and two simulators using ModbusTool)
Here are my configuration files


I then used MQTTfx to connect to the mosquito instance using the pi's IP address and port 1883 and the default user/pw
Username: ClassicClient
Password: ClassicClient123

Published {"Wake"} to the "ClassicMQTT/workshop/cmnd" and "ClassicMQTT/dev/cmnd" topics to get it to send the readings...

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 26, 2021, 10:45:18 AM
Thanks.

If I reboot my PI do I need to start Portainer again somehow?  I had it working, then rebooted and not I cannot pull it up.  Does it run as a service?

I am using a plugin for my  Homeseer automation server called MCSmqtt.  I works really slick.  Easy to get the data I want into automation. 
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 26, 2021, 11:01:14 AM
Quote from: ForeverHomestead on January 26, 2021, 10:45:18 AM
Thanks.

If I reboot my PI do I need to start Portainer again somehow?  I had it working, then rebooted and not I cannot pull it up.  Does it run as a service?

I am using a plugin for my  Homeseer automation server called MCSmqtt.  I works really slick.  Easy to get the data I want into automation.

It runs as a docker image. Use the command

docker ps

to list all the running containers, if it's not there, use the following to start

docker start portainer-ce


that plugin does look really slick
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 26, 2021, 11:39:55 PM
Quote from: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8

I have been trying to follow this and do the build.
I get an error though when I try to do the final build ( and I am using all his latest revisions) .
Did you get it working using this method  ?
I mainly want to just use the IOTstack mosquitto and not the other one installed by ClassicMQTT for pi .

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 27, 2021, 07:12:34 AM
Larry,    Mine is working,  but I didn't yet go through the remaining tweaks in the writeup to clean up a couple things better.    Mainly the environment variables instead of hardcoding the command line args.

What error do you get?  Post the error in the gist comments and Paraphraser will probably pipe in, especially if you link a copy of your docker-compose.yml file in pastebin or comments.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on January 27, 2021, 05:10:55 PM
Well, I just read the comments Paraphraser and I am not excited. I put alot of work into that code and the snarky comments were not good. I am considering removing the python version from the public.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ForeverHomestead on January 27, 2021, 06:00:26 PM
When MQTT publishes info like this

{"BatTemperature":0.8,
  "NetAmpHours":4,
  "ChargeState":0,
  "InfoFlagsBits":-1308610300,
  "ReasonForResting":4,
  "NegativeAmpHours":-9369,
  "BatVoltage":25.9,
  "PVVoltage":10.5,
  "VbattRegSetPTmpComp":27.4,
  "TotalAmpHours":704,
  "WhizbangBatCurrent":-0.5,
  "BatCurrent":0.0,
  "PVCurrent":0.0,
  "ConnectionState":0,
  "EnergyToday":0.5,
  "EqualizeTime":10800,
  "SOC":100,
  "Aux1":false,
  "Aux2":false,
  "Power":0.0,
  "FETTemperature":14.0,
  "PositiveAmpHours":17138,
  "TotalEnergy":616.1,
  "FloatTimeTodaySeconds":29689,
  "RemainingAmpHours":704,
  "AbsorbTime":18000,
  "ShuntTemperature":4.0,
  "PCBTemperature":21.3}


What format is the "AbsorbTime":18000, in.

I am trying to figure out how to get it into useful format in my .net script.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 27, 2021, 06:41:47 PM
Quote from: mcsarge on January 27, 2021, 05:10:55 PM
Well, I just read the comments Paraphraser and I am not excited. I put alot of work into that code and the snarky comments were not good. I am considering removing the python version from the public.

Matt -
I appreciate all the work you did on getting the classic mqtt working. It has been really stable and works. I ventured into the IOTStack as an easy way to load those other programs like grafana . As far as Paraprhaser's build - it just doesn't work . I turned the other builds off and turned your builds on - they work.  I ventured into the IOTstack version and stuff just as a way to learn more about docker .  I didn't understand why he criticized the comments - it just seems like common sense and teaching others and I am sure most appreciate that you  took the time to put them in.
My only goal was to have one version of Mosquitto running - and I probably can - I just haven't quite figured it all out yet.  Please accept my apologies if anything I did on my end contributed to your frustration about this.
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 27, 2021, 09:07:43 PM
Matt,   Thank you for ClassicMQTT!   It did work already as it was, and your instructions are clear and easy enough for a middle aged hack to follow!   I set it up and manually started the docker image (method 2), and it worked.  It's awesome, I'm already using the bat voltage measured from the Classic in my system control a little bit, charge another battery bank when the main battery is not accepting any more AC coupled solar.     

My Pi I already had IOTstack on, I don't think I would have gotten all of mosquitto / nodered / influx / grafana all working together otherwise  (middle aged hack and all...).  I didn't know how to combine all that and have ClassicMQTT set up running from docker-compose and started on bootup, have it backed up, not have double mosquitto.  I think all the items need to be under one docker so they all share their same little network within docker?   Anyway, after a bunch of back & forth on discord, I got mine working, sort of temporary in a couple of details though.  None of my trouble with is the fault of your code or setup, only my lack of expertise with docker, and the complexity of all the other shtuff on the Pi.   I'm planning to redo it to see if I understand all the details, and test it some more.  I was planning to offer another install method description in the wiki, ie: method 4, if that's of interest.
Thanks,  Dale

Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 28, 2021, 08:25:47 AM
Quote from: ForeverHomestead on January 27, 2021, 06:00:26 PM
...

What format is the "AbsorbTime":18000, in.

I am trying to figure out how to get it into useful format in my .net script.

It's the value in register 4139
you can find it in the attached classic_register_map document

(http://capture.png)
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on January 28, 2021, 08:29:06 AM
I appreciate the support guys, but those comments were way off base. I posted a note in there that he should have just started with the simple docker image and then he would not had to have figured out what I was doing with mosquitto.


Matt

Quote from: daklein on January 27, 2021, 09:07:43 PM
Matt,   Thank you for ClassicMQTT!   It did work already as it was, and your instructions are clear and easy enough for a middle aged hack to follow!   I set it up and manually started the docker image (method 2), and it worked.  It's awesome, I'm already using the bat voltage measured from the Classic in my system control a little bit, charge another battery bank when the main battery is not accepting any more AC coupled solar.     

My Pi I already had IOTstack on, I don't think I would have gotten all of mosquitto / nodered / influx / grafana all working together otherwise  (middle aged hack and all...).  I didn't know how to combine all that and have ClassicMQTT set up running from docker-compose and started on bootup, have it backed up, not have double mosquitto.  I think all the items need to be under one docker so they all share their same little network within docker?   Anyway, after a bunch of back & forth on discord, I got mine working, sort of temporary in a couple of details though.  None of my trouble with is the fault of your code or setup, only my lack of expertise with docker, and the complexity of all the other shtuff on the Pi.   I'm planning to redo it to see if I understand all the details, and test it some more.  I was planning to offer another install method description in the wiki, ie: method 4, if that's of interest.
Thanks,  Dale
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 28, 2021, 11:47:32 AM
Quote from: ClassicCrazy on January 26, 2021, 11:39:55 PM
Quote from: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8

I have been trying to follow this and do the build.
I get an error though when I try to do the final build ( and I am using all his latest revisions) .
Did you get it working using this method  ?
I mainly want to just use the IOTstack mosquitto and not the other one installed by ClassicMQTT for pi .

Larry
I'm not sure if I understand what he's trying to do, in step 5 he's modifying the docker-compose.yml file which is the output of IOTStacks ./menu.sh "Build Stack" command.

IOTStack allows you to add a custom service using the compose-override.yml file which will get merged into the generated docker-compose.yml file when you run the "Build Stack" command.
The resulting docker-compose.yml file will include the content of the compose-override.yml, the image: classicdiy/classicmqtt in the classic_mqtt: section tells docker compose to get the image from dockerHub.
You also have to create a .env file that has the parameters for the classic.
See "Adding custom services" in: https://sensorsiot.github.io/IOTstack/Custom/ (https://sensorsiot.github.io/IOTstack/Custom/)

I created an organization and repository on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt that has the docker image for the Raspberry Pi 4 ARM.
All you need from the classicMQTT github page is the compose-override.yml file (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml))
Place this file in the IOTStack directory on the pi along with the .env file.

I added a page to the wiki that describes the installation steps using IOTStack https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack), let me know if I missed any steps.
I can also add how to setup multiple classics using IOTStack if anyone is interested...

Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 28, 2021, 12:02:43 PM
Quote from: Graham on January 28, 2021, 11:47:32 AM
Quote from: ClassicCrazy on January 26, 2021, 11:39:55 PM
Quote from: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8

I have been trying to follow this and do the build.
I get an error though when I try to do the final build ( and I am using all his latest revisions) .
Did you get it working using this method  ?
I mainly want to just use the IOTstack mosquitto and not the other one installed by ClassicMQTT for pi .

Larry
I'm not sure if I understand what he's trying to do, in step 5 he's modifying the docker-compose.yml file which is the output of IOTStacks ./menu.sh "Build Stack" command.

IOTStack allows you to add a custom service using the compose-override.yml file which will get merged into the generated docker-compose.yml file when you run the "Build Stack" command.
The resulting docker-compose.yml file will include the content of the compose-override.yml, the image: classicdiy/classicmqtt in the classic_mqtt: section tells docker compose to get the image from dockerHub.
You also have to create a .env file that has the parameters for the classic.
See "Adding custom services" in: https://sensorsiot.github.io/IOTstack/Custom/ (https://sensorsiot.github.io/IOTstack/Custom/)

I created an organization and repository on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt that has the docker image for the Raspberry Pi 4 ARM.
All you need from the classicMQTT github page is the compose-override.yml file (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml))
Place this file in the IOTStack directory on the pi along with the .env file.

I added a page to the wiki that describes the installation steps using IOTStack https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack), let me know if I missed any steps.
I can also add how to setup multiple classics using IOTStack if anyone is interested...

Graham.

I am giving this a try following your instructions . I will let you know how it goes.
I deleted the IOTstack classic mqtt that was there so I start out fresh I hope .
And yes I would like to know how to set up the multiple classics.
Thanks
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 28, 2021, 01:53:15 PM
Quote from: ClassicCrazy on January 28, 2021, 12:02:43 PM
...
I am giving this a try following your instructions . I will let you know how it goes.
I deleted the IOTstack classic mqtt that was there so I start out fresh I hope .
And yes I would like to know how to set up the multiple classics.
Thanks
Larry

You'll have to delete the docker containers and images
use docker ps to list the containers
and docker images to list the images
then sudo docker rm <containerId> to stop each container
and sudo docker rmi <imageId> to delete the images

you could also uninstall docker to really start out fresh

use dpkg -l | grep -i docker to list all the docker packages you have on your pi
then sudo apt remove docker-c* to uninstall each docker-c... package

you probably have
docker-ce
docker-ce-cli
docker-compose
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 28, 2021, 03:53:36 PM
Quote from: Graham on January 28, 2021, 01:53:15 PM
Quote from: ClassicCrazy on January 28, 2021, 12:02:43 PM
...
I am giving this a try following your instructions . I will let you know how it goes.
I deleted the IOTstack classic mqtt that was there so I start out fresh I hope .
And yes I would like to know how to set up the multiple classics.
Thanks
Larry

You'll have to delete the docker containers and images
use docker ps to list the containers
and docker images to list the images
then sudo docker rm <containerId> to stop each container
and sudo docker rmi <imageId> to delete the images

you could also uninstall docker to really start out fresh

use dpkg -l | grep -i docker to list all the docker packages you have on your pi
then sudo apt remove docker-c* to uninstall each docker-c... package

you probably have
docker-ce
docker-ce-cli
docker-compose

Okay - I think I will do that - nothing is working out too well. I was tempted to just reflash my pi card and start out fresh . But I will give your suggestions a try.
Do I need to wipe out my old Classic-Mqtt that runs under Python too ?
I tried so many things in the last few days I think I might be better off just starting with fresh flash of pi

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on January 28, 2021, 06:51:10 PM
Larry,

I have wiped and started again with my Raspberry Pi so many times, they are uncountable. There are ways to remove everything but etching a new SDcard is just easier - I sometime create a new SDCard and save the old one, you can put it in a USB reader and stick it in the newly programmed Pi and get anything you might have left behind.


[/quote]

Okay - I think I will do that - nothing is working out too well. I was tempted to just reflash my pi card and start out fresh . But I will give your suggestions a try.
Do I need to wipe out my old Classic-Mqtt that runs under Python too ?
I tried so many things in the last few days I think I might be better off just starting with fresh flash of pi

Larry
[/quote]
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 28, 2021, 07:40:47 PM
Quote from: mcsarge on January 28, 2021, 06:51:10 PM
Larry,

I have wiped and started again with my Raspberry Pi so many times, they are uncountable. There are ways to remove everything but etching a new SDcard is just easier - I sometime create a new SDCard and save the old one, you can put it in a USB reader and stick it in the newly programmed Pi and get anything you might have left behind.



Okay - I think I will do that - nothing is working out too well. I was tempted to just reflash my pi card and start out fresh . But I will give your suggestions a try.
Do I need to wipe out my old Classic-Mqtt that runs under Python too ?
I tried so many things in the last few days I think I might be better off just starting with fresh flash of pi

Larry
[/quote]
[/quote]

Good advice Matt  on saving the old card - next time I will do that . I just wiped it and am starting out fresh which I am getting pretty good at setting them back up again now !

Larry


Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on January 28, 2021, 11:10:00 PM
Quote from: Graham on January 28, 2021, 11:47:32 AM

I created an organization and repository on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt that has the docker image for the Raspberry Pi 4 ARM.
All you need from the classicMQTT github page is the compose-override.yml file (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml))
Place this file in the IOTStack directory on the pi along with the .env file.

I added a page to the wiki that describes the installation steps using IOTStack https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack), let me know if I missed any steps.
I can also add how to setup multiple classics using IOTStack if anyone is interested...

Graham.

Thanks Graham,   I plan to try installing it this way also.   Are the dockerhub images specific to the Pi hardware?  ie:  3b vs. 4 ?         Dale
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 28, 2021, 11:26:00 PM
Quote from: Graham on January 28, 2021, 11:47:32 AM
Quote from: ClassicCrazy on January 26, 2021, 11:39:55 PM
Quote from: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8

I have been trying to follow this and do the build.
I get an error though when I try to do the final build ( and I am using all his latest revisions) .
Did you get it working using this method  ?
I mainly want to just use the IOTstack mosquitto and not the other one installed by ClassicMQTT for pi .

Larry
I'm not sure if I understand what he's trying to do, in step 5 he's modifying the docker-compose.yml file which is the output of IOTStacks ./menu.sh "Build Stack" command.

IOTStack allows you to add a custom service using the compose-override.yml file which will get merged into the generated docker-compose.yml file when you run the "Build Stack" command.
The resulting docker-compose.yml file will include the content of the compose-override.yml, the image: classicdiy/classicmqtt in the classic_mqtt: section tells docker compose to get the image from dockerHub.
You also have to create a .env file that has the parameters for the classic.
See "Adding custom services" in: https://sensorsiot.github.io/IOTstack/Custom/ (https://sensorsiot.github.io/IOTstack/Custom/)

I created an organization and repository on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt that has the docker image for the Raspberry Pi 4 ARM.
All you need from the classicMQTT github page is the compose-override.yml file (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml))
Place this file in the IOTStack directory on the pi along with the .env file.

I added a page to the wiki that describes the installation steps using IOTStack https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack), let me know if I missed any steps.
I can also add how to setup multiple classics using IOTStack if anyone is interested...

Graham.

I installed Pi OS  Buster full version with suggested software on Pi3B+ with 32 GB sd card

sudo apt install -y git curl  was already installed on my pi

It didn't build the classic_mqtt - I first  had copied wrong file( classic_mqtt_compose.yml) and did build which didn't work.  Next I deleted that and copied compose-override.yaml  into Iotstack  and did build again, then restarted stack
It is not building the classic-mqtt container
I tried a few times and restarted pi too.
update - found problem - - I had named compose-override.yaml instead of .yml   - geez !
Now it built okay
success on using MQTT.fx
( need to really follow those screen shots carefully )
success on using connecting to ClassicMQTT  using Node Red ( Node Red is built into new Raspberry Pi builds - at least mine so I used that instead of installing from IOTstack)
No success so far into getting Android Classic Monitoring App to connect via MQTT - I will revisit the wiki to see if I am doing something wrong.
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 29, 2021, 12:29:47 AM
Quote from: ClassicCrazy on January 28, 2021, 11:26:00 PM
Quote from: Graham on January 28, 2021, 11:47:32 AM
Quote from: ClassicCrazy on January 26, 2021, 11:39:55 PM
Quote from: daklein on January 26, 2021, 01:01:14 AM

This would be very helpful for anyone using classicMQTT with an IOTstack RPi, me for one!  Big thanks to Paraphraser from IOTstack discord!

https://gist.github.com/Paraphraser/378d17e759901d4871e5be4327e3f3c8

I have been trying to follow this and do the build.
I get an error though when I try to do the final build ( and I am using all his latest revisions) .
Did you get it working using this method  ?
I mainly want to just use the IOTstack mosquitto and not the other one installed by ClassicMQTT for pi .

Larry
I'm not sure if I understand what he's trying to do, in step 5 he's modifying the docker-compose.yml file which is the output of IOTStacks ./menu.sh "Build Stack" command.

IOTStack allows you to add a custom service using the compose-override.yml file which will get merged into the generated docker-compose.yml file when you run the "Build Stack" command.
The resulting docker-compose.yml file will include the content of the compose-override.yml, the image: classicdiy/classicmqtt in the classic_mqtt: section tells docker compose to get the image from dockerHub.
You also have to create a .env file that has the parameters for the classic.
See "Adding custom services" in: https://sensorsiot.github.io/IOTstack/Custom/ (https://sensorsiot.github.io/IOTstack/Custom/)

I created an organization and repository on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt that has the docker image for the Raspberry Pi 4 ARM.
All you need from the classicMQTT github page is the compose-override.yml file (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml (https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml))
Place this file in the IOTStack directory on the pi along with the .env file.

I added a page to the wiki that describes the installation steps using IOTStack https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack), let me know if I missed any steps.
I can also add how to setup multiple classics using IOTStack if anyone is interested...

Graham.



I started over and installed fresh  Pi OS  Buster full version with suggested software on Pi3B+ with 32 GB sd card

sudo apt install -y git curl  was already installed on my pi

It didn't build the classic_mqtt - I first  had copied wrong file( classic_mqtt_compose.yml) and did build which didn't work.  Next I deleted that and copied compose-override.yaml  into Iotstack  and did build again, then restarted stack
It is not building the classic-mqtt container
I tried a few times and restarted pi too.
update - found problem - - I had named compose-override.yaml instead of .yml   - geez !
Now it built okay
success on using MQTT.fx and connecting
( need to really follow those screen shots from directions carefully  )
success on using connecting to ClassicMQTT  using Node Red ( Node Red is built into new Raspberry Pi builds - at least mine so I used that instead of installing from IOTstack)
success on getting Android Classic Monitoring  App on MQTT 
( first had trouble getting mqtt working on my Fire android tablet - but after clearing , uninstalling, installing newest version , more stuff like restarting it I finally got it to work . Version says 3.0.6 . but I installed same APK on my galaxy 4 phone and it shows 3.1.0 version. )
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 29, 2021, 08:52:15 AM
Quote from: ClassicCrazy on January 29, 2021, 12:29:47 AM
...
success on getting Android Classic Monitoring  App on MQTT 
( first had trouble getting mqtt working on my Fire android tablet - but after clearing , uninstalling, installing newest version , more stuff like restarting it I finally got it to work . Version says 3.0.6 . but I installed same APK on my galaxy 4 phone and it shows 3.1.0 version. )
Larry

Hmm, not sure why it's showing version 3.1.0, could your galaxy 4 already have had version 3.1.0, you would not be able to install 3.0.6 without first uninstalling 3.1.0.
The "MQTT" branch of ClassicMonitor is currently at version 3.0.6,
the new "master" branch (currently 3.1.4) is the merge of the latest updates of the app store branch "NoMQTT" with the MQTT branch as well as an experimental update to access my Tri-Star TS-45 using the modbus adapter.
Version 3.0.6 should be ok for testing ClassicMQTT


Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 29, 2021, 09:39:53 AM
Quote from: daklein on January 28, 2021, 11:10:00 PM
...

Thanks Graham,   I plan to try installing it this way also.   Are the dockerhub images specific to the Pi hardware?  ie:  3b vs. 4 ?         Dale

Looks like Larry got it working on a Pi3B+

I have it working on Docker Desktop on Windows 10 as well
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 29, 2021, 11:48:02 AM
Quote from: Graham on January 29, 2021, 08:52:15 AM
Quote from: ClassicCrazy on January 29, 2021, 12:29:47 AM
...
success on getting Android Classic Monitoring  App on MQTT 
( first had trouble getting mqtt working on my Fire android tablet - but after clearing , uninstalling, installing newest version , more stuff like restarting it I finally got it to work . Version says 3.0.6 . but I installed same APK on my galaxy 4 phone and it shows 3.1.0 version. )
Larry

Hmm, not sure why it's showing version 3.1.0, could your galaxy 4 already have had version 3.1.0, you would not be able to install 3.0.6 without first uninstalling 3.1.0.
The "MQTT" branch of ClassicMonitor is currently at version 3.0.6,
the new "master" branch (currently 3.1.4) is the merge of the latest updates of the app store branch "NoMQTT" with the MQTT branch as well as an experimental update to access my Tri-Star TS-45 using the modbus adapter.
Version 3.0.6 should be ok for testing ClassicMQTT


Graham

I probably had some other version of Classic Monitoring app on my phone already and didn't realize it- I don't use that phone too often as I have lineage OS on it and don't pay for phone service.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 31, 2021, 10:41:49 AM
I updated the image on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt (https://hub.docker.com/r/classicdiy/classicmqtt) so that you can now define the startup parameters in the environment section of the docker-compose.yml file when using IOTStack to setup your docker containers. You no longer need the .env file or the command: parameters.
Updated the wiki accordingly...
https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack)


Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 31, 2021, 01:24:58 PM
Quote from: Graham on January 31, 2021, 10:41:49 AM
I updated the image on dockerHub https://hub.docker.com/r/classicdiy/classicmqtt (https://hub.docker.com/r/classicdiy/classicmqtt) so that you can now define the startup parameters in the environment section of the docker-compose.yml file when using IOTStack to setup your docker containers. You no longer need the .env file or the command: parameters.
Updated the wiki accordingly...
https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack)


Graham

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3

To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

Duplicate keys will become an error in future releases, and are errors
by default when using the new API.

I had already deleted .env file , then I deleted the classic part of docker_compose.yml , I used portainer and removed previous classicmqtt image, rebooted pi , but it still won't build and I get the same error as above.
After breakfast I may try to do what the link in the error message says to do if I can figure out how to do that.

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 31, 2021, 02:42:12 PM
Quote from: ClassicCrazy on January 31, 2021, 01:24:58 PM

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=wt9m']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3
...
Larry

Looks like you have the container name in there twice

the second block should be called something different than the first and must have a unique container_name (see attached example)

you can delete your docker-compose.yml file in IOTStack, the menu.sh->Build Stack will create a new one with everything you selected.


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 31, 2021, 03:00:55 PM
Quote from: Graham on January 31, 2021, 02:42:12 PM
Quote from: ClassicCrazy on January 31, 2021, 01:24:58 PM

I got this error on build - probably because I had previous build the other way ?
Issue running build:
while constructing a mapping
  in "./compose-override.yml", line 2, column 3
found duplicate key "classic_mqtt" with value "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.2.31', 'CLASSIC_PORT=502', 'CLASSIC_NAME=HoopPV', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=wt9m']), ('networks', ['iotstack_nw'])])" (original value: "ordereddict([('container_name', 'classic_mqtt'), ('image', 'classicdiy/classicmqtt'), ('restart', 'unless-stopped'), ('environment', ['LOGLEVEL=DEBUG', 'CLASSIC=192.168.3.2', 'CLASSIC_PORT=502', 'CLASSIC_NAME=CLASSIC2', 'MQTT_HOST=mosquitto', 'MQTT_PORT=1883', 'MQTT_ROOT=ClassicMQTT', 'MQTT_USER=ClassicPublisher', 'MQTT_PASS=ClassicPub']), ('networks', ['iotstack_nw'])])")
  in "./compose-override.yml", line 20, column 3
...
Larry

Looks like you have the container name in there twice

the second block should be called something different than the first and must have a unique container_name (see attached example)

you can delete your docker-compose.yml file in IOTStack, the menu.sh->Build Stack will create a new one with everything you selected.


Graham.

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on January 31, 2021, 03:08:19 PM
Quote from: ClassicCrazy on January 31, 2021, 03:00:55 PM
...

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry

thanks Larry
I updated the file


Graham
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: ClassicCrazy on January 31, 2021, 03:18:04 PM
Quote from: Graham on January 31, 2021, 03:08:19 PM
Quote from: ClassicCrazy on January 31, 2021, 03:00:55 PM
...

That seems to have done it Graham - thanks.
You may want to edit the file I got off the link .
This one - is where I got it from
https://github.com/ClassicDIY/ClassicMQTT/blob/master/code/Python/compose-override.yml

Larry

thanks Larry
I updated the file


Graham

Okay rebuilt worked .
I wasn't getting Classic2 publishing
Just found a typo I put in on the ip address
That took awhile for me to figure out !
Larry
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 16, 2021, 08:35:48 AM
Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on February 16, 2021, 08:52:10 AM
Quote from: mcsarge on February 16, 2021, 08:35:48 AM
Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt

Hey Matt, did you see my pull request https://github.com/ClassicDIY/ClassicMQTT/pull/25 (https://github.com/ClassicDIY/ClassicMQTT/pull/25) that adds the environment variables...


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: mcsarge on February 16, 2021, 05:20:40 PM
Well,

No I did not, stupid me. I put a bunch of work into the validator set them first and your method of setting them in the initialization looks far easier! I will get your changes and then push them down to me.

Matt

Quote from: Graham on February 16, 2021, 08:52:10 AM
Quote from: mcsarge on February 16, 2021, 08:35:48 AM
Hey All,

I am working on an update to the python implementation, it can read the environment variables directly in the code, so once the environment variable are set in the docker-compose or at the command line, it will be able to get those values in the python code. I am implementing it so it will use the environment variables if the value is not set in the command line. That way a command line setting will override the environment variable. I think the the whole creating of the .env file confused too many people. As soon as I get it finished and tested I will create a pull.

This may make the Home Assistance setup easier too.

Matt

Hey Matt, did you see my pull request https://github.com/ClassicDIY/ClassicMQTT/pull/25 (https://github.com/ClassicDIY/ClassicMQTT/pull/25) that adds the environment variables...


Graham.
Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on May 10, 2021, 05:57:09 PM
Thanks McSarge & Graham for this great tool.  I have some limited data available from my inverters, but with classicmqtt I'm also able to get battery bank voltage, which makes it possible to roughly control some loads and make use of variable solar production, instead of losing it as the main battery is near full.

I redid some of my RPi with ClassicMQTT & IOTstack, after the sd card filled up and things broke.   I believe I had previously built the docker image & was using the .env & command line argument method to give ClassicMQTT some of the parameters.  It worked. 

I followed these instructions (Thanks!),  https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack
This time I think I am running the image from DockerHub, portainer shows me that it was built 1/31   
classicdiy/classicmqtt:latest   120.3 MB   2021-01-31 08:56:57

It is working, but,  the former .env file & command line arguments for publish rate,  now in the environment section of compose-override.yml seem to not have an effect.     It publishes just at the default 5 minute rate, I'm not sure what I'm missing?  I'm pretty sure it's seeing the compose-override.yml file I'm editing,  if I change the classic's ip addr, etc, that will break it or not.   

If i change the override file, and run the ./menu.sh script to build and start the IOTstack,  it does recreate the classicmqtt container, and the main docker-compose.yml gets text changes.


Start Stack:
docker-compose up -d --remove-orphans
WARNING: Some networks were defined but are not used by any service: vpn_nw, iotstack_nw_internal, nextcloud_internal
nodered is up-to-date
portainer-ce is up-to-date
grafana is up-to-date
influxdb is up-to-date
Recreating classic_mqtt ...
mosquitto is up-to-date
Recreating classic_mqtt ... done

Stack Started




Here's what's in the top of docker-compose.yml,  from the compose-override.yml file that was edited.

pi@hassbian:~/IOTstack $ more docker-compose.yml
version: '3.6'
services:
  classic_mqtt:
    container_name: classic_mqtt
    image: classicdiy/classicmqtt
    restart: unless-stopped
    environment:
    - LOGLEVEL=DEBUG
    - CLASSIC=192.168.16.10
    - CLASSIC_PORT=502
    - CLASSIC_NAME=MNC250CP
    - MQTT_HOST=mosquitto
    - MQTT_PORT=1883
    - MQTT_ROOT=ClassicMQTT
    - MQTT_USER=mqtt_user
    - MQTT_PASS=mqtt_user_pw
    - AWAKE_PUBLISH_RATE=3
    - SNOOZE_PUBLISH_RATE=5
    networks:
    - iotstack_nw

# uncomment the following block if you have a second classic, change the CLASSIC ip address and CLASSIC_NAME
#  classic_mqtt2:
#    container_name: classic_mqtt2
#    image: classicdiy/classicmqtt
#    restart: unless-stopped
#    environment:
#      - LOGLEVEL=DEBUG
#      - CLASSIC=192.168.86.137
#      - CLASSIC_PORT=502
#      - CLASSIC_NAME=MyShed
#      - MQTT_HOST=mosquitto
#      - MQTT_PORT=1883
#      - MQTT_ROOT=ClassicMQTT
#      - MQTT_USER=ClassicPublisher
#      - MQTT_PASS=ClassicPub123
#    networks:
#      - iotstack_nw

# Here is an example using an external MQTT broker
#  classic_mqtt:
#    container_name: classic_mqtt
#    image: classicdiy/classicmqtt
#    restart: unless-stopped
#    environment:
#      - LOGLEVEL=DEBUG
#      - CLASSIC=192.168.86.37
#      - CLASSIC_PORT=502
#      - CLASSIC_NAME=MyWorkshop
#      - MQTT_HOST=192.168.86.82
#      - MQTT_PORT=1883
#      - MQTT_ROOT=ClassicMQTT
#      - MQTT_USER=HASSUser
#      - MQTT_PASS=HASSpw
#    networks:
#      - iotstack_nw
  grafana:
    container_name: grafana
    image: grafana/grafana
    restart: unless-stopped
    user: "0"
    ports:
    - "3000:3000"
    environment:
    - GF_PATHS_DATA=/var/lib/grafana
    - GF_PATHS_LOGS=/var/log/grafana
    volumes:
    - ./volumes/grafana/data:/var/lib/grafana
    - ./volumes/grafana/log:/var/log/grafana
    networks:
    - iotstack_nw

  python:
    container_name: python
    build: ./services/python/.
    restart: unless-stopped
    network_mode: host
    volumes:
    - ./volumes/python/app:/usr/src/app
  portainer-ce:

.  etcetera
.      etcetera
.           etcetera



Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on May 11, 2021, 10:56:59 AM
Quote from: daklein on May 10, 2021, 05:57:09 PM
...
It is working, but,  the former .env file & command line arguments for publish rate,  now in the environment section of compose-override.yml seem to not have an effect.     It publishes just at the default 5 minute rate, I'm not sure what I'm missing?  I'm pretty sure it's seeing the compose-override.yml file I'm editing,  if I change the classic's ip addr, etc, that will break it or not.   


I pushed a new image to dockerHub, try re running "Build Stack" and "Start Stack" in IOTStack/menu.sh
I also had to set the SNOOZE_PUBLISH_RATE to 300

check the logs in portainer for classic_mqtt to verify the startup arguments

let me know if this works...
Graham.


Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: daklein on May 11, 2021, 05:48:50 PM
Quote from: Graham on May 11, 2021, 10:56:59 AM

I pushed a new image to dockerHub, try re running "Build Stack" and "Start Stack" in IOTStack/menu.sh
I also had to set the SNOOZE_PUBLISH_RATE to 300

check the logs in portainer for classic_mqtt to verify the startup arguments

let me know if this works...
Graham.

Graham,   Thanks!    It does work.  It got the new image from dockerhub, and it's publishing at the faster rate I was looking for.

I set both the wake & snooze interval to 3 sec, and it seems to be populating into influxdb at that rate now.

The log shows it going in & out of snooze mode, opening and closing the connection to the Classic.  Is that ok, or likely to cause some issue?   Is snooze meant to be nighttime, no solar production, for time > WAKE_DURATION.
   
(An issue beyond the need to downsample longer term data in database, which I need to figure out eventually.  But I want the relatively faster rate to use for control, using battery voltage vs. target voltage.)

2021-05-11 21:31:07,934:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:07,935:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:07,938:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings
2021-05-11 21:31:07,940:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/info
2021-05-11 21:31:13,920:DEBUG:classic_mqtt:Opening the modbus Connection
2021-05-11 21:31:13,940:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:13,941:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:13,945:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings
2021-05-11 21:31:13,949:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/info
2021-05-11 21:31:19,923:DEBUG:classic_mqtt:Opening the modbus Connection
2021-05-11 21:31:19,942:DEBUG:classic_mqtt:Closing the modbus Connection, we are in Snooze mode
2021-05-11 21:31:19,944:DEBUG:classic_mqtt:Got data from Classic at 192.168.16.10:502
2021-05-11 21:31:19,948:DEBUG:classic_mqtt:Publishing: ClassicMQTT/MNC250CP/stat/readings



Title: Re: ESP32 Classic MQTT Publisher & Home Assistant
Post by: Graham on May 12, 2021, 08:45:33 AM
Quote from: daklein on May 11, 2021, 05:48:50 PM
...
The log shows it going in & out of snooze mode, opening and closing the connection to the Classic.  Is that ok, or likely to cause some issue?   Is snooze meant to be nighttime, no solar production, for time > WAKE_DURATION.
   

SNOOZE_PUBLISH_RATE should be around 300 seconds since, in this mode, it will disconnect from modbus after every read to allow the local app to get in and access the classic.
If you want it to stay awake you can send a MQTT cmnd with a json payload containing "stayAwake"

ClassicMQTT/MNC250CP/cmnd {"stayAwake"}