Using IOTstack, Telegraph, Graphana and MQTT Classic data

Started by ClassicCrazy, December 05, 2021, 03:03:52 PM

Previous topic - Next topic

ClassicCrazy

I made a post about how to install Node Red on Raspberry Pi ( using new Bullseye OS)
Graham suggested trying using Telegraph to send data directly to Graphana via his project  Classic DIY  MQTT software .
With his help I got it to work - thanks Graham !
It is documented in this post.
http://midniteftp.com/forum/index.php?topic=5738.0
Also - remember to restart Docker stacks after making changes to .conf files.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

boB

K7IQ 🌛  He/She/Me

ClassicCrazy

Quote from: boB on December 05, 2021, 05:01:03 PM

That is some cool looking interfaces, eh ?
Yes
That is just what Graham provided for the sample he shared. There are so many options for graphics in Grafana and should even be easy enough that even I can figure it out !
I will play around with it later and add some more fields for different data.
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

#3
Here is my latest Grafana
I am still working on it. Easy to switch these numerical panels back to graphs if wanted.
Grafana is well documented so I have been able to google search any issues I have had .
For example - The total amps or watts output from the two separate Classics would not give a result.
I figured out that was because the simple a + b equation would not work since the two datas were different time stamps.
But there is a way to resample the last data to make the math addition work .
I have been going about learning Grafana piece meal instead of watching a basic beginner course. And so far that works just playing around with it.
When I take a detailed beginner lesson I think it will sink in more.
For some reason I haven't been able to get the absorb or float time to work . Not sure why - getting an Influx error on that like the data name isn't correct .
Edit - fixed the Absorb and Float times - I had double parenthesis on one end and that was all it took to mess it up !  select  AbsorbTime  FROM "mqtt_consumer" WHERE ("topic" = 'ClassicMQTT/CLASSIC2/stat/readings'))
Anyway here is my latest screenshot and also the dashboard code if anyone wants to plug it in and try it out.
You would need to go through each setting and give it the name of your own Classic ( that is after you have all the other associated MQTT stuff set up) .
So far this is my favorite Classic Monitor since I can just leave it running on the Pi and display it on big screen of my PC monitor.
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

boB

K7IQ 🌛  He/She/Me

ClassicCrazy

Quote from: boB on December 18, 2021, 04:51:08 PM

That looks great, Larry !

boB
Now I am going to try and tackle getting  Node Red to read from the same MQTT Pi source - so I can hopefully make it control other things.
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

#6
I played around with the dashboard and jazzed it up with some nicer colors and figured out how to do graphing better than I had been.
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

A few tweaks and sharing the Grafana  dashboard graphic  .
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry

could you use one of the grafana plugins like traffic light or status panel?

https://grafana.com/grafana/plugins/?type=panel


Graham.
Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Quote from: Graham on December 23, 2021, 12:40:25 PM
Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry

could you use one of the grafana plugins like traffic light or status panel?

https://grafana.com/grafana/plugins/?type=panel


Graham.

I will check that out Graham.

Last night I also have worked out having node red take the MQTT Aux data , then used a function and change node to change the true  and false into a number. I just need to figure out how to send it to Influxdb under a new database. Got close last night but not quite there .

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

I got the Absorb and Float timers working ( counting down )  in Grafana
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

#11
Quote from: Graham on December 23, 2021, 12:40:25 PM
Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
could you use one of the grafana plugins like traffic light or status panel?
https://grafana.com/grafana/plugins/?type=panel
Graham.

Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
I found this note
"JSON
The JSON data format parses a JSON object or an array of objects into metric fields.
NOTE: All JSON numbers are converted to float fields. JSON strings and booleans are ignored unless specified in the tag_key or json_string_fields options."
https://github.com/influxdata/telegraf/tree/master/plugins/parsers/json

  ## Array of glob pattern strings or booleans keys that should be added as string fields.
  json_string_fields = []

I sort of solved this
I just had to add this line at bottom of telegraf.conf

## Data format to consume.
  ## Each data format has it's own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "json"
  json_string_fields = ["ClassicMQTT/#"]

But for some reason Aux1 will report through but not Aux2 .
Learning a lot along the way .
 
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Graham

Quote from: ClassicCrazy on December 24, 2021, 10:07:10 PM

Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
...

Good catch,
You can just add the following to the telegraf.conf file so that telegraf includes the boolean fields

json_string_fields = ["Aux1", "Aux2"]

If you connect to the influx container and run the 'influx' command, you can show the field keys on the mqtt_solar database to see the fieldKeys now include Aux1 and 2

commands:
influx
show field keys on mqtt_solar



Graham.

Off-Grid Island cottage, Lac Simon QC Canada
370 Ahrs @ 24V (4 8L16 batteries)
4 x 250watt panels on dual axis trackers http://tinyurl.com/hfpkgr5
Classic 150, Whizbang Jr.
Android Monitor: http://tinyurl.com/lomzq3s

ClassicCrazy

Quote from: Graham on December 26, 2021, 10:16:49 AM
Quote from: ClassicCrazy on December 24, 2021, 10:07:10 PM

Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
...

Good catch,
You can just add the following to the telegraf.conf file so that telegraf includes the boolean field
json_string_fields = ["Aux1", "Aux2"]
If you connect to the influx container and run the 'influx' command, you can show the field keys on the mqtt_solar database to see the fieldKeys now include Aux1 and 2
commands:
influx
show field keys on mqtt_solar
Graham.
Adding that line did the trick Graham
I appreciate your screenshots because now I know how to use portainer to check logs and get into the console to check on things like in influx above.
Saves a thousand words.
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

ClassicCrazy

#14
Here is a quick video tour of my solar system and the  messing around I have been doing on the solar system and monitoring.
https://youtu.be/qj97LjGcj1Y
Yes I know my controls wiring is nothing fancy and kind of messy but stuff gets done and easy to modify on the fly.
I just got this device to monitor the direct DC
https://www.aliexpress.com/item/32914076246.html?spm=a2g0s.12269583.0.0.77366f33ZuNvrN
And also the AC version one
https://www.aliexpress.com/item/32872623242.html?spm=a2g0s.12269583.0.0.343a2028zJMxDY
And also got another updated Sonoff POW R2 which i can put on individual circuit if I want .
https://www.ebay.com/itm/384229223436
Plenty to keep busy with on these cold winter days ( though warming up enough this week to make the hookups for peacefair montitors) .
Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable