MQTT for the android app

Started by Graham, January 27, 2019, 03:50:42 PM

Previous topic - Next topic

Graham

Well it's cold outside so I decided to crack open the android code for the classic monitor app. I added support for MQTT (both publisher & subscriber) so that you can access the classics information from the internet without having to port forward/ddns your router (but it does require a second android device).

Still a work in progress and might have a few issues but here is a link to my dropbox folder for those who would like to try it out

https://www.dropbox.com/sh/z3kzddtj17vk9t2/AACFwHN0phXpMuD9T3-Kvt8Ta?dl=0
might have to uninstall the old version first.
this is a one click install if you have the dropbox app on your device.

Here is a bit of documentation on the new feature

http://graham22.github.io/Classic/classicmonitor/help_en.html#MQTT

not sure if I'll release this to google play since there is a lot of testing that has not been done
-the subscriber currently does not support the logs or calendar
-I haven't tested it with more than one classic (or system view)
-haven't tested the impact on PVOutput
-settings upgrade from previous release not implemented


also working on a ESP32 implementation of the publisher which I'll probably post on my github page at some point.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

boB

Thank you Graham !

You really do wonderful work !

boB
K7IQ 🌛  He/She/Me

ClassicCrazy

Thanks Graham - maybe tomorrow I will mess around with it and try mqtt - thanks for your efforts at making this better and better !
Will also look forward to your esp32 developments. In case you haven't run across these two guys
Andreas Spiess has done some good evaulations and experiments with ESP32 boards
https://www.youtube.com/channel/UCu7_D0o48KbfhpEohoP7YSQ/videos
Also G6EJD does a lot of with ESP32
https://www.youtube.com/user/G6EJD/videos

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

That'd be great if you could try it, let me know if you get it to work

it's still a little quirky
here are some of the steps

- setup the publisher first
- uninstall your current version before installing the new one since the settings might not migrate
- let it connect to your classic as usual
- enter your MQTT credentials as a publisher (from the email from dioty.co)
- restart the app & see that it connects to the classic & you get the SOC tab...

- on a second device, install the app
- change the settings to not auto detect classic
- remove any connected classics from the nav bar by sliding them to the right in the event it connected before you disabled auto connect
- enter your MQTT credentials as a Subscriber
- restart the app

you can use MQTTfx to see the data being published to the broker from a windows pc
subscribe to:
/'ROOT_TOPIC'/classic/stat/#
&
/'ROOT_TOPIC'/classic/cmnd/#

where 'ROOT_TOPIC' is whats in the dioty.co email, in my case it was

Your root topic: "/graham.a.ross@gmail.com/"
omit the "

you can also use a local MQTT broker on a raspberry Pi for testing etc. I used Mosquito

Graham

Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Getting ready to install this on an android and try it out.
On another note - do you think there would be a way to have the app add up two different Classics in Follow Me mode to show the total output  ?
Right now I have two shunts and Whizbangs for each Classic but I have all the power going through the first Classic Shunt so that I can see the total system amps going into battery . To see the true output for both Classics I need to toggle between the two and add up their outputs.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

I got it working Graham .
I set up mqtt account
Installed your beta app on one android phone and then set up as publisher as  you documented.
I then installed beta app on second android phone and set it up as subscriber.
At first I wasn't getting anything on second phone.
I have both phones rooted and running Lineage builds. They have a Privacy Guard for apps - I messed around with that on both androids trying to give it permissions or turn Privacy Guard off and then restarted second android and finally saw MQTT listed in the Network and when I selected that it worked and I see the Classic info via mqtt
I don't seem to be able to select the second Classic I have on the first android connected to network - I see it but it won't connect to it .
I just restarted first android to see how it effects second android and second android responds soon as I start classic on first android.

Interesting - I just restarted second android ( subscriber ) and it is showing the name of the second Classic but showing the data from the other Classic on Publisher android . Still can't get Publisher Android to show any data from second Classic on my system.

But anyway it works so I have it all set up to test any updates now .

Larry 
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: ClassicCrazy on January 30, 2019, 04:47:09 PM
I got it working Graham ...

That's great Larry, you should be able to access your data from the internet now...

The code isn't setup to handle more than one classic, I'll probably have to generate unique MQTT topics for each classic, not sure how to do that when in 'System View'.
I'm currently finishing up the ESP32 implementation of the publisher.
also, I was thinking that it would't be to difficult for someone to implement a Node-Red implementation of the publisher for the Kid so they could use the android app as a subscriber...

here is a bit of background info about the app;

The publisher app subscribes to /'root_topic'/classic/cmnd/wake & /'root_topic'/classic/cmnd/info and publishes the following topics /'root_topic'/classic/stats/info & /'root_topic'/classic/stats/readings.

The 'wake' or 'info' command will cause the publisher to publish 'readings' every second for a minute, if it doesn't get another wake it will publish once every 5 minutes.
The 'info' command will cause the publisher to issue one info payload below

when the subscriber starts it will send a /'root_topic'/classic/cmnd/info with the payload {"info"} to the broker and a 'wake' once a minute.


/'root_topic'/classic/cmnd/info json payload example


   "appVersion":"2193",
   "buildDate":"Tuesday, February 6, 2018",
   "deviceName":"Workshop",
   "deviceType":"Classic",
   "endingAmps":10.8,
   "hasWhizbang":true,
   "lastVOC":5.0,
   "model":"Classic 200 (rev 4)",
   "mpptMode":11,
   "netVersion":"2122",
   "nominalBatteryVoltage":12,
   "unitID":-1901764051
}

/'root_topic'/classic/cmnd/readings json payload example

{
   "BatTemperature":18.3,
   "NetAmpHours":0,
   "ChargeState":0,
   "InfoFlagsBits":-1577045760,
   "ReasonForResting":104,
   "NegativeAmpHours":1331,
   "BatVoltage":12.0,
   "PVVoltage":5.2,
   "VbattRegSetPTmpComp":14.8,
   "TotalAmpHours":63,
   "WhizbangBatCurrent":-0.3,
   "BatCurrent":0.0,
   "PVCurrent":0.0,
   "ConnectionState":0,
   "EnergyToday":0.0,
   "EqualizeTime":7200,
   "SOC":100,
   "Aux1":false,
   "Aux2":false,
   "Power":0.0,
   "FETTemperature":26.2,
   "PositiveAmpHours":1457,
   "TotalEnergy":89.2,
   "FloatTimeTodaySeconds":0,
   "RemainingAmpHours":63,
   "AbsorbTime":7200,
   "ShuntTemperature":22.0,
   "PCBTemperature":34.7
}


Graham
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

I am glad you know how to figure all that out Graham.
I noticed there were limits on the free account . Would using your mqtt app ever exceed the free limit ?
I haven't had it on other than trying it out.
If I am following you correctly - are you saying that I could use a Raspberry Pi with it's built in Node Red to gather and publish the data from the Classic in format of your app so that an android phone could then display it on your Classic Monitoring app in subscribe mode ?
That  sounds like a great solution if a Pi Zero W would do the job - only one watt of power needed ( other than the wireless router ) .

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: ClassicCrazy on February 01, 2019, 09:16:29 PM
I am glad you know how to figure all that out Graham.
I noticed there were limits on the free account . Would using your mqtt app ever exceed the free limit ?
I haven't had it on other than trying it out.
If I am following you correctly - are you saying that I could use a Raspberry Pi with it's built in Node Red to gather and publish the data from the Classic in format of your app so that an android phone could then display it on your Classic Monitoring app in subscribe mode ?
That  sounds like a great solution if a Pi Zero W would do the job - only one watt of power needed ( other than the wireless router ) .

Larry

I think the limit is 50,000 messages per month, the publisher will post every second up to a minute if it doesn't get a 'Wake" command at which point it will slow the message rate to post once every 5 minutes so I guess it depend on how long you use the subscriber, it should be ok if you just check it once in a while.
The AWS IoT Core service charges $1 per one million messages but I haven't tried setting up an Amazon account yet.

Node-Red: you just have to publish the same json payload in my previous message and it should work with the subscriber app for any controller as long as you can get the data from off the controller either through modbus for the classic or the com port for the kid...

I'm still tinkering with the ESP32, looking at https://www.blynk.cc/ as an alternative to the app...


Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Wow - lots of options for the Blynk app.
Seems like enough people with coding experience have tried this and that monitoring options over the years.
Maybe something like Blynk can bring a group collaboration ?  I would help if I knew how to do that , I only know enough to get by and figure out what is being done. I guess like someone who can understand a foreign language but can't speak it.

If you start the Blynk route  maybe make a new topic on it and see if anyone else will help and contribute.

Happy to test anything out !

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Here is a link to my github page for the ESP32 MQTT/Blynk publisher

https://github.com/graham22/ClassicMQTT

The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK


Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Graham ,
I watched this last night and thought maybe you would find some interest in it .
Seems like it could be useful for dealing with large amounts of data and displaying it.
https://youtu.be/JdV4x925au0

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher

https://github.com/graham22/ClassicMQTT

The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK


Graham.

I will try to take a look at this later and see if I can get it going.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher

https://github.com/graham22/ClassicMQTT

The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK


Graham.

Graham
Can you give me a bit more info on this ESP8266 setup ?
I have one that I just loaded up ESP Easy Mega on .
What do I do with your code - is it an add on to ESP Easy or is it an independent flash to the esp8266 ?

Thanks
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: ClassicCrazy on March 07, 2019, 02:24:22 PM
Quote from: Graham on March 01, 2019, 10:05:51 AM
Here is a link to my github page for the ESP32 MQTT/Blynk publisher

https://github.com/graham22/ClassicMQTT

The blynk implementation is limited to 6 gadgets due to the 'free' limit.
The ESP can be configured to use MQTT and/or BLYNK


Graham.

Graham
Can you give me a bit more info on this ESP8266 setup ?
I have one that I just loaded up ESP Easy Mega on .
What do I do with your code - is it an add on to ESP Easy or is it an independent flash to the esp8266 ?

Thanks
Larry

There is a flash tool in the dist folder once you download and extract the zip from espeasy
https://github.com/letscontrolit/ESPEasy/tree/mega/dist

unfortunately it didn't work when I tried the latest flashEsp8266.exe, the documentation says it should work with the ESP32 dev board but I get an error at the end of flashing.

the other way is to load and build the project in the arduino ide, target the ESP32 dev board and install all the required libraries used in the project (#include ...)

I'll see if I can find another flash tool since getting all the required libraries and building the project from scratch might take a while...


Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s