Well it's cold outside so I decided to crack open the android code for the classic monitor app. I added support for MQTT (both publisher & subscriber) so that you can access the classics information from the internet without having to port forward/ddns your router (but it does require a second android device).
Still a work in progress and might have a few issues but here is a link to my dropbox folder for those who would like to try it out
https://www.dropbox.com/sh/z3kzddtj17vk9t2/AACFwHN0phXpMuD9T3-Kvt8Ta?dl=0 (https://www.dropbox.com/sh/z3kzddtj17vk9t2/AACFwHN0phXpMuD9T3-Kvt8Ta?dl=0)
might have to uninstall the old version first.
this is a one click install if you have the dropbox app on your device.
Here is a bit of documentation on the new feature
http://graham22.github.io/Classic/classicmonitor/help_en.html#MQTT (http://graham22.github.io/Classic/classicmonitor/help_en.html#MQTT)
not sure if I'll release this to google play since there is a lot of testing that has not been done
-the subscriber currently does not support the logs or calendar
-I haven't tested it with more than one classic (or system view)
-haven't tested the impact on PVOutput
-settings upgrade from previous release not implemented
also working on a ESP32 implementation of the publisher which I'll probably post on my github page at some point.
Thank you Graham !
You really do wonderful work !
boB
Thanks Graham - maybe tomorrow I will mess around with it and try mqtt - thanks for your efforts at making this better and better !
Will also look forward to your esp32 developments. In case you haven't run across these two guys
Andreas Spiess has done some good evaulations and experiments with ESP32 boards
https://www.youtube.com/channel/UCu7_D0o48KbfhpEohoP7YSQ/videos
Also G6EJD does a lot of with ESP32
https://www.youtube.com/user/G6EJD/videos
Larry
That'd be great if you could try it, let me know if you get it to work
it's still a little quirky
here are some of the steps
- setup the publisher first
- uninstall your current version before installing the new one since the settings might not migrate
- let it connect to your classic as usual
- enter your MQTT credentials as a publisher (from the email from dioty.co)
- restart the app & see that it connects to the classic & you get the SOC tab...
- on a second device, install the app
- change the settings to not auto detect classic
- remove any connected classics from the nav bar by sliding them to the right in the event it connected before you disabled auto connect
- enter your MQTT credentials as a Subscriber
- restart the app
you can use MQTTfx to see the data being published to the broker from a windows pc
subscribe to:
/'ROOT_TOPIC'/classic/stat/#
&
/'ROOT_TOPIC'/classic/cmnd/#
where 'ROOT_TOPIC' is whats in the dioty.co email, in my case it was
Your root topic: "/graham.a.ross@gmail.com/"
omit the "
you can also use a local MQTT broker on a raspberry Pi for testing etc. I used Mosquito
Graham
Getting ready to install this on an android and try it out.
On another note - do you think there would be a way to have the app add up two different Classics in Follow Me mode to show the total output ?
Right now I have two shunts and Whizbangs for each Classic but I have all the power going through the first Classic Shunt so that I can see the total system amps going into battery . To see the true output for both Classics I need to toggle between the two and add up their outputs.
Larry
I got it working Graham .
I set up mqtt account
Installed your beta app on one android phone and then set up as publisher as you documented.
I then installed beta app on second android phone and set it up as subscriber.
At first I wasn't getting anything on second phone.
I have both phones rooted and running Lineage builds. They have a Privacy Guard for apps - I messed around with that on both androids trying to give it permissions or turn Privacy Guard off and then restarted second android and finally saw MQTT listed in the Network and when I selected that it worked and I see the Classic info via mqtt
I don't seem to be able to select the second Classic I have on the first android connected to network - I see it but it won't connect to it .
I just restarted first android to see how it effects second android and second android responds soon as I start classic on first android.
Interesting - I just restarted second android ( subscriber ) and it is showing the name of the second Classic but showing the data from the other Classic on Publisher android . Still can't get Publisher Android to show any data from second Classic on my system.
But anyway it works so I have it all set up to test any updates now .
Larry
Quote from: ClassicCrazy on January 30, 2019, 04:47:09 PM
I got it working Graham ...
That's great Larry, you should be able to access your data from the internet now...
The code isn't setup to handle more than one classic, I'll probably have to generate unique MQTT topics for each classic, not sure how to do that when in 'System View'.
I'm currently finishing up the ESP32 implementation of the publisher.
also, I was thinking that it would't be to difficult for someone to implement a Node-Red implementation of the publisher for the Kid so they could use the android app as a subscriber...
here is a bit of background info about the app;
The publisher app subscribes to /'root_topic'/classic/cmnd/wake & /'root_topic'/classic/cmnd/info and publishes the following topics /'root_topic'/classic/stats/info & /'root_topic'/classic/stats/readings.
The 'wake' or 'info' command will cause the publisher to publish 'readings' every second for a minute, if it doesn't get another wake it will publish once every 5 minutes.
The 'info' command will cause the publisher to issue one info payload below
when the subscriber starts it will send a /'root_topic'/classic/cmnd/info with the payload {"info"} to the broker and a 'wake' once a minute.
/'root_topic'/classic/cmnd/info json payload example
{
"appVersion":"2193",
"buildDate":"Tuesday, February 6, 2018",
"deviceName":"Workshop",
"deviceType":"Classic",
"endingAmps":10.8,
"hasWhizbang":true,
"lastVOC":5.0,
"model":"Classic 200 (rev 4)",
"mpptMode":11,
"netVersion":"2122",
"nominalBatteryVoltage":12,
"unitID":-1901764051
}
/'root_topic'/classic/cmnd/readings json payload example
{
"BatTemperature":18.3,
"NetAmpHours":0,
"ChargeState":0,
"InfoFlagsBits":-1577045760,
"ReasonForResting":104,
"NegativeAmpHours":1331,
"BatVoltage":12.0,
"PVVoltage":5.2,
"VbattRegSetPTmpComp":14.8,
"TotalAmpHours":63,
"WhizbangBatCurrent":-0.3,
"BatCurrent":0.0,
"PVCurrent":0.0,
"ConnectionState":0,
"EnergyToday":0.0,
"EqualizeTime":7200,
"SOC":100,
"Aux1":false,
"Aux2":false,
"Power":0.0,
"FETTemperature":26.2,
"PositiveAmpHours":1457,
"TotalEnergy":89.2,
"FloatTimeTodaySeconds":0,
"RemainingAmpHours":63,
"AbsorbTime":7200,
"ShuntTemperature":22.0,
"PCBTemperature":34.7
}
Graham
I am glad you know how to figure all that out Graham.
I noticed there were limits on the free account . Would using your mqtt app ever exceed the free limit ?
I haven't had it on other than trying it out.
If I am following you correctly - are you saying that I could use a Raspberry Pi with it's built in Node Red to gather and publish the data from the Classic in format of your app so that an android phone could then display it on your Classic Monitoring app in subscribe mode ?
That sounds like a great solution if a Pi Zero W would do the job - only one watt of power needed ( other than the wireless router ) .
Larry
Quote from: ClassicCrazy on February 01, 2019, 09:16:29 PM
I am glad you know how to figure all that out Graham.
I noticed there were limits on the free account . Would using your mqtt app ever exceed the free limit ?
I haven't had it on other than trying it out.
If I am following you correctly - are you saying that I could use a Raspberry Pi with it's built in Node Red to gather and publish the data from the Classic in format of your app so that an android phone could then display it on your Classic Monitoring app in subscribe mode ?
That sounds like a great solution if a Pi Zero W would do the job - only one watt of power needed ( other than the wireless router ) .
Larry
I think the limit is 50,000 messages per month, the publisher will post every second up to a minute if it doesn't get a 'Wake" command at which point it will slow the message rate to post once every 5 minutes so I guess it depend on how long you use the subscriber, it should be ok if you just check it once in a while.
The AWS IoT Core service charges $1 per one million messages but I haven't tried setting up an Amazon account yet.
Node-Red: you just have to publish the same json payload in my previous message and it should work with the subscriber app for any controller as long as you can get the data from off the controller either through modbus for the classic or the com port for the kid...
I'm still tinkering with the ESP32, looking at https://www.blynk.cc/ (https://www.blynk.cc/) as an alternative to the app...
Graham.
Wow - lots of options for the Blynk app.
Seems like enough people with coding experience have tried this and that monitoring options over the years.
Maybe something like Blynk can bring a group collaboration ? I would help if I knew how to do that , I only know enough to get by and figure out what is being done. I guess like someone who can understand a foreign language but can't speak it.
If you start the Blynk route maybe make a new topic on it and see if anyone else will help and contribute.
Happy to test anything out !
Larry
Here is a link to my github page for the ESP32 MQTT/Blynk publisher
https://github.com/graham22/ClassicMQTT (https://github.com/graham22/ClassicMQTT)
The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK
Graham.
Graham ,
I watched this last night and thought maybe you would find some interest in it .
Seems like it could be useful for dealing with large amounts of data and displaying it.
https://youtu.be/JdV4x925au0
Larry
Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher
https://github.com/graham22/ClassicMQTT (https://github.com/graham22/ClassicMQTT)
The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK
Graham.
I will try to take a look at this later and see if I can get it going.
Larry
Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher
https://github.com/graham22/ClassicMQTT (https://github.com/graham22/ClassicMQTT)
The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK
Graham.
Graham
Can you give me a bit more info on this ESP8266 setup ?
I have one that I just loaded up ESP Easy Mega on .
What do I do with your code - is it an add on to ESP Easy or is it an independent flash to the esp8266 ?
Thanks
Larry
Quote from: ClassicCrazy on March 07, 2019, 02:24:22 PM
Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher
https://github.com/graham22/ClassicMQTT (https://github.com/graham22/ClassicMQTT)
The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK
Graham.
Graham
Can you give me a bit more info on this ESP8266 setup ?
I have one that I just loaded up ESP Easy Mega on .
What do I do with your code - is it an add on to ESP Easy or is it an independent flash to the esp8266 ?
Thanks
Larry
There is a flash tool in the dist folder once you download and extract the zip from espeasy
https://github.com/letscontrolit/ESPEasy/tree/mega/dist (https://github.com/letscontrolit/ESPEasy/tree/mega/dist)
unfortunately it didn't work when I tried the latest flashEsp8266.exe, the documentation says it should work with the ESP32 dev board but I get an error at the end of flashing.
the other way is to load and build the project in the arduino ide, target the ESP32 dev board and install all the required libraries used in the project (#include ...)
I'll see if I can find another flash tool since getting all the required libraries and building the project from scratch might take a while...
Graham.
So I just flash your bin file ?
I will give that a try .
The flash tool I used today from espeasy was a lot fanicer than their old tool
Larry
I put all the flash tools from espeasy in the ClassicMQTT that I downloaded from your github .
When I open up flash tool it does not find a bin file to flash. So I went back your github page and found a .bin file and downloaded that and placed it in the ClassicMQTT folder but flash tool still won't find it. If I open up your .bin folder it does not show anything. If I go to the ESPEASY mega folder for their flash - there is all kinds of .bin files in the Bin folder.
So I believe something is wrong with your Bin folder or format that it won't show up. It does not say it is a zip file so I did not unzip it - just placed it in the folder as it was .
Update - I found that I just needed to put the .bin file in the Bin folder and I think it is flashing okay now .
I flashed it - but I don't know what I am supposed to do next. I can't seem to connect to it on serial port. I tried flashing again with flash tool and put in my wifi settings but don't see it show up .
I tried just about every baud setting to try and get into it on serial terminal program via computer but nothing shows anything legible
Larry
Quote from: ClassicCrazy on March 07, 2019, 10:40:38 PM
...
Update - I found that I just needed to put the .bin file in the Bin folder and I think it is flashing okay now .
I flashed it - but I don't know what I am supposed to do next. I can't seem to connect to it on serial port. I tried flashing again with flash tool and put in my wifi settings but don't see it show up .
I tried just about every baud setting to try and get into it on serial terminal program via computer but nothing shows anything legible
Larry
Are you using the ESP32 or the ESP8266?
the bin file is compiled for the ESP32 since the 8266 doesn't have enough space for the OTA feature
https://www.amazon.ca/Lidahaotin-ESP-WROOM-32-Development-Dual-Mode-Bluetooth/dp/B07JVNVS8D/ref=sr_1_4?keywords=esp32+wroom+development+board&qid=1552048619&s=gateway&sr=8-4 (https://www.amazon.ca/Lidahaotin-ESP-WROOM-32-Development-Dual-Mode-Bluetooth/dp/B07JVNVS8D/ref=sr_1_4?keywords=esp32+wroom+development+board&qid=1552048619&s=gateway&sr=8-4)
I was able to flash the ESP32 using the expressif flash tool http://iot-bits.com/esp32/esp32-flash-download-tool-tutorial (http://iot-bits.com/esp32/esp32-flash-download-tool-tutorial)
Once it is flashed and rebooted you should see it setup as an access point in the serial terminal, once you connect to the access point, open the web page at 192.168.4.1 and click 'Configure WIFI' to setup the wifi credentials, classic IP/Port, Blynk token and the MQTT credentials.
Graham.
oh yeah I used an esp8266 - have to get an ESP32 . Have been holding off on getting one of those since every time I get something it is outdated by a better version - ha ha !
Larry
Graham,
I am trying to compile your ClassicMQTT code and I am having a tough time getting all the libraries and the correct versions. Do you happen to have a listing of the ones you are using?
Also, in the Classic App for MQTT, the length of the hostname is pretty short - I am hosting an MQTT server on Azure, and the host name is quite long. Any chance of opening that up for more characters? Lastly in the Classic App, the length of the field that holds the port number is short too - I use nGrok to forward port 502 outside the cellular firewall, but the port number assigned is 1 character longer than your app allows for the port number.
Matt
Hi Matt,
I'm updating the code to use a more recent library, I'll post an update later today...
My development environment is
Visual Studio Code with the PlatformIO extension
it will download and install all dependencies defined in the platform.ini file
Graham.
Graham,
I will look into using PlatformIO in VSC.
I have a Raspberry Pi local to my Classic (on a remote island in Canada) and so I am working on a python implementation of the MODBUS-to-MQTT tool. I have setup a server in the cloud using my Azure $150/month credit and have install MQTT on it and plan to have the python code publish to that.
I will need to implement the feature that you have that allows a subscriber to notify the publisher that someone is listening and so publish at a faster rate.
Many thanks for all this work! If I get the python code working, should I add it to your repository? Let me know.
Matt
Matt,
I checked in an update to the ClassicMQTT github repo that uses a more recent Wifi library (IOTWebConf) plus an async MQTT client.
I tested it with my android app that can subscribe to mqtt,
The ESP code will start in AP mode for 30 seconds then switch to STATION if an SSID/PW is configured
The root topic is the 'Thing Name' in the setup.
Graham.
(http://mqtt.png)
Any idea what the password to the AP is so I can configure the device?
Matt
Quote from: mcsarge on December 18, 2019, 11:06:13 PM
Any idea what the password to the AP is so I can configure the device?
Matt
If it is like the other esp8266 I have used you probably don't need a password - just leave it blank or enter one of your own when you put in your wireless info. Then look for the address it connects to on the wifi and connect up to it directly - you would be able to configure it there. But not sure if this is like the others.
I dug out an esp32 board and might give this a try in the next day or two. Kind of late here now. I am happy because I just made a CI-V control cable for my Icom 706MKIIG using just a usb to ttl board , a resistor and diode and it works ! I am trying out the digital modes . Anyway I had to boast about my electronics ability here - ha ha . 10 minutes to make it, but a few hours of digging around on the internet on how to make one.
Larry
It should be 'ClassicMQTT'
Connect to the 'ClassicMQTT' access point
browse to 192.168.4.1
Once wifi is setup you can access the config page by browsing to the IP of the esp32 with 'admin' username and 'ClassicMQTT' as pw (or whatever you put in "AP Password")
more info is available at https://github.com/prampec/IotWebConf (https://github.com/prampec/IotWebConf)
After examining the code, the initial password is ClassicMQTT.
I was able to get the esp32 to connect to my Classic using ngrok (I had to expand the digits allowed for saving the port number) and also connect to my MQTT server.
I got some weird numbers being posted to the MQTT server. Any thoughts on this?
{
"BatTemperature": 6536.31,
"NetAmpHours": 0,
"ChargeState": 0,
"InfoFlagsBits": -1308610300,
"ReasonForResting": 104,
"NegativeAmpHours": -8539,
"BatVoltage": 24.91,
"PVVoltage": 9.41,
"VbattRegSetPTmpComp": 30.6,
"TotalAmpHours": 536,
"WhizbangBatCurrent": 6553.01,
"BatCurrent": 0.01,
"PVCurrent": 0.01,
"ConnectionState": 0,
"EnergyToday": 0.01,
"EqualizeTime": 10800,
"SOC": 98,
"Aux1": false,
"Aux2": false,
"Power": 0.01,
"FETTemperature": 6541.61,
"PositiveAmpHours": 16014,
"TotalEnergy": 581.91,
"FloatTimeTodaySeconds": 0,
"RemainingAmpHours": 529,
"AbsorbTime": 18000,
"ShuntTemperature": -14.99,
"PCBTemperature": 6550.01
}
hmm...
I had to comment out the NetAmpHours but the other values look ok
I'm getting this in MQTT.fx subscribed to the ClassicMQTT/classic/stat/readings topic
{
"BatTemperature": 18.81,
"NetAmpHours": 0,
"ChargeState": 0,
"InfoFlagsBits": -1577046016,
"ReasonForResting": 104,
"NegativeAmpHours": -1399,
"BatVoltage": 13.71,
"PVVoltage": 16.11,
"VbattRegSetPTmpComp": 13.6,
"TotalAmpHours": 63,
"WhizbangBatCurrent": 0.21,
"BatCurrent": 0.71,
"PVCurrent": 0.61,
"ConnectionState": 0,
"EnergyToday": 0.11,
"EqualizeTime": 7200,
"SOC": 100,
"Aux1": false,
"Aux2": false,
"Power": 8.01,
"FETTemperature": 32.11,
"PositiveAmpHours": 2000,
"TotalEnergy": 103.91,
"FloatTimeTodaySeconds": 40421,
"RemainingAmpHours": 63,
"AbsorbTime": 7200,
"ShuntTemperature": 24.01,
"PCBTemperature": 40.01
}
It looks like the 1 byte values are OK, but the 2 byte values are messed up. Battery Temp? It is negative for me now, so maybe it is that your code is not accounting for 2's complement negative numbers?
Matt
Here is how I fixed this. The number in temperature is a signed 16 bit integer, so convert it to a signed int first, then divide it by the factor:
float GetFloatValue(int index, uint8_t* data, float div = 1.0) {
index *= 2;
int16_t temp = (data[index] << 8 | data[index + 1]);
return temp/div;
}
Matt
ok thanks Matt,
I updated github with that change
let me know your github username and I'll add you as a collaborator
mcsarge is my GitHub user.
I have changes for the size of the stored port as well. I would also like to look into storing either a URL or an IP for the Classic. When I use ngrok to get port 502 from out behind the cellular system firewall, I get an address like tcp://0.tcp.ngrok.io:12345" and the host IP is dynamic so I can look up the IP for the host but it changes.
Matt
I invited you as a collaborator to the ClassicMQTT repository
+ I re-added the support for blynk and a few other fixes so you'll have to do a pull to sync my updates
Are you planning to put the ESP on the same local network as the classic so you won't have to port forward 502?
Graham.
Graham,
Ultimately, I might put the ESP in the same network, but right now the island is under a few inches of snow and ice locked so nobody is getting to it until Spring. To be honest, I will probably take the code for your esp and convert it to a python app to run on my Raspberry Pi which is local to the Classic and already internet connected and it will post to my server on Azure or to a Mosquitto install on my local Pi - not sure yet, but i do not want to use a ton of data up on my cellular connection keeping the MQTT updated off-island.
Matt
Hey Graham,
I am trying to build the Classic MQTT on an Adafruit ESP32 Feather - which is just an ESP32 board and I keep getting some errors in Platformio which seem to indicate that there is a mismatch in libraries. Have you seen this before? Any suggestions?
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp: In member function 'bool AsyncClient::connect(IPAddress, uint16_t)':
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp:115:8: error: 'ip_addr_t {aka struct ip_addr}' has no
member named 'addr'
addr.addr = ip;
^
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp: In member function 'bool AsyncClient::connect(const char*, uint16_t)':
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp:148:35: error: 'ip_addr_t {aka struct ip_addr}' has no member named 'addr'
return connect(IPAddress(addr.addr), port);
^
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp: In member function 'bool AsyncClient::operator==(const AsyncClient&)':
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp:192:66: error: 'ip_addr_t {aka struct ip_addr}' has no member named 'addr'
Quote from: mcsarge on February 18, 2020, 05:00:40 PM
Hey Graham,
I am trying to build the Classic MQTT on an Adafruit ESP32 Feather - which is just an ESP32 board and I keep getting some errors in Platformio which seem to indicate that there is a mismatch in libraries. Have you seen this before? Any suggestions?
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp: In member function 'bool AsyncClient::connect(IPAddress, uint16_t)':
.pio\libdeps\featheresp32\ESPAsyncTCP_ID305\src\ESPAsyncTCP.cpp:115:8: error: 'ip_addr_t {aka struct ip_addr}' has no
...
ESPAsyncTCP is a dependant of AsyncMQTTClient
here are the versions that platformIO uses on my system (attached image)
if you look your libdeps folder, do you have the same?
have you changed the platform.ini board or platform
Graham,
In my lib_deps I just have the library names, I did not use specific versions, I will change to use the versions and I am sure that will fix it.
Many thanks,
Matt
Quote from: mcsarge on February 19, 2020, 07:48:58 AM
Graham,
In my lib_deps I just have the library names, I did not use specific versions, I will change to use the versions and I am sure that will fix it.
Many thanks,
Matt
hmm, I didn't have to specify the versions, I just deleted everything under .pio\libdeps\esp32dev then clicked build
build output attached
I did the same thing a couple of times and still get build problem. Maybe I have a library installed that is over-riding the most recent version of a library or something. I will give it another go.
Matt