I made a post about how to install Node Red on Raspberry Pi ( using new Bullseye OS)
Graham suggested trying using Telegraph to send data directly to Graphana via his project Classic DIY MQTT software .
With his help I got it to work - thanks Graham !
It is documented in this post.
http://midniteftp.com/forum/index.php?topic=5738.0
Also - remember to restart Docker stacks after making changes to .conf files.
Larry
That is some cool looking interfaces, eh ?
Quote from: boB on December 05, 2021, 05:01:03 PM
That is some cool looking interfaces, eh ?
Yes
That is just what Graham provided for the sample he shared. There are so many options for graphics in Grafana and should even be easy enough that even I can figure it out !
I will play around with it later and add some more fields for different data.
Larry
Here is my latest Grafana
I am still working on it. Easy to switch these numerical panels back to graphs if wanted.
Grafana is well documented so I have been able to google search any issues I have had .
For example - The total amps or watts output from the two separate Classics would not give a result.
I figured out that was because the simple a + b equation would not work since the two datas were different time stamps.
But there is a way to resample the last data to make the math addition work .
I have been going about learning Grafana piece meal instead of watching a basic beginner course. And so far that works just playing around with it.
When I take a detailed beginner lesson I think it will sink in more.
For some reason I haven't been able to get the absorb or float time to work . Not sure why - getting an Influx error on that like the data name isn't correct .
Edit - fixed the Absorb and Float times - I had double parenthesis on one end and that was all it took to mess it up ! select AbsorbTime FROM "mqtt_consumer" WHERE ("topic" = 'ClassicMQTT/CLASSIC2/stat/readings'))
Anyway here is my latest screenshot and also the dashboard code if anyone wants to plug it in and try it out.
You would need to go through each setting and give it the name of your own Classic ( that is after you have all the other associated MQTT stuff set up) .
So far this is my favorite Classic Monitor since I can just leave it running on the Pi and display it on big screen of my PC monitor.
Larry
That looks great, Larry !
boB
Quote from: boB on December 18, 2021, 04:51:08 PM
That looks great, Larry !
boB
Now I am going to try and tackle getting Node Red to read from the same MQTT Pi source - so I can hopefully make it control other things.
Larry
I played around with the dashboard and jazzed it up with some nicer colors and figured out how to do graphing better than I had been.
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
A few tweaks and sharing the Grafana dashboard graphic .
Larry
Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
could you use one of the grafana plugins like traffic light or status panel?
https://grafana.com/grafana/plugins/?type=panel (https://grafana.com/grafana/plugins/?type=panel)
Graham.
Quote from: Graham on December 23, 2021, 12:40:25 PM
Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
could you use one of the grafana plugins like traffic light or status panel?
https://grafana.com/grafana/plugins/?type=panel (https://grafana.com/grafana/plugins/?type=panel)
Graham.
I will check that out Graham.
Last night I also have worked out having node red take the MQTT Aux data , then used a function and change node to change the true and false into a number. I just need to figure out how to send it to Influxdb under a new database. Got close last night but not quite there .
Larry
I got the Absorb and Float timers working ( counting down ) in Grafana
Larry
Quote from: Graham on December 23, 2021, 12:40:25 PM
Quote from: ClassicCrazy on December 20, 2021, 04:55:00 PM
I still haven't figured out how to take some data like false or true and have it display so I can make some on or off status indicators for the AUX .
Larry
could you use one of the grafana plugins like traffic light or status panel?
https://grafana.com/grafana/plugins/?type=panel (https://grafana.com/grafana/plugins/?type=panel)
Graham.
Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
I found this note
"JSON
The JSON data format parses a JSON object or an array of objects into metric fields.
NOTE: All JSON numbers are converted to float fields. JSON strings and booleans are ignored unless specified in the tag_key or json_string_fields options."
https://github.com/influxdata/telegraf/tree/master/plugins/parsers/json
## Array of glob pattern strings or booleans keys that should be added as string fields.
json_string_fields = []
I sort of solved this
I just had to add this line at bottom of telegraf.conf
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "json"
json_string_fields = ["ClassicMQTT/#"]
But for some reason Aux1 will report through but not Aux2 .
Learning a lot along the way .
Larry
Quote from: ClassicCrazy on December 24, 2021, 10:07:10 PM
Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
...
Good catch,
You can just add the following to the telegraf.conf file so that telegraf includes the boolean fields
json_string_fields = ["Aux1", "Aux2"]
If you connect to the influx container and run the 'influx' command, you can show the field keys on the mqtt_solar database to see the fieldKeys now include Aux1 and 2
commands:
influx
show field keys on mqtt_solar
Graham.
Quote from: Graham on December 26, 2021, 10:16:49 AM
Quote from: ClassicCrazy on December 24, 2021, 10:07:10 PM
Graham ,
I think the reason I can't get the Aux display to work is because Telegraf is not passing Boolean
...
Good catch,
You can just add the following to the telegraf.conf file so that telegraf includes the boolean field
json_string_fields = ["Aux1", "Aux2"]
If you connect to the influx container and run the 'influx' command, you can show the field keys on the mqtt_solar database to see the fieldKeys now include Aux1 and 2
commands:
influx
show field keys on mqtt_solar
Graham.
Adding that line did the trick Graham
I appreciate your screenshots because now I know how to use portainer to check logs and get into the console to check on things like in influx above.
Saves a thousand words.
Larry
Here is a quick video tour of my solar system and the messing around I have been doing on the solar system and monitoring.
https://youtu.be/qj97LjGcj1Y
Yes I know my controls wiring is nothing fancy and kind of messy but stuff gets done and easy to modify on the fly.
I just got this device to monitor the direct DC
https://www.aliexpress.com/item/32914076246.html?spm=a2g0s.12269583.0.0.77366f33ZuNvrN
And also the AC version one
https://www.aliexpress.com/item/32872623242.html?spm=a2g0s.12269583.0.0.343a2028zJMxDY
And also got another updated Sonoff POW R2 which i can put on individual circuit if I want .
https://www.ebay.com/itm/384229223436
Plenty to keep busy with on these cold winter days ( though warming up enough this week to make the hookups for peacefair montitors) .
Larry
Quote from: ClassicCrazy on January 11, 2022, 01:39:31 PM
Here is a quick video tour of my solar system and the messing around I have been doing on the solar system and monitoring.
https://www.ebay.com/itm/384229223436
...
Larry, I don't see the link to the video, the first link is to ebay (same as the last link)
Graham
Quote from: Graham on January 12, 2022, 08:25:07 AM
Quote from: ClassicCrazy on January 11, 2022, 01:39:31 PM
Here is a quick video tour of my solar system and the messing around I have been doing on the solar system and monitoring.
...
Larry, I don't see the link to the video, the first link is to ebay (same as the last link)
Graham
Graham -
Yeah I goofed up the link
Here it is - also I corrected it in the other post.
https://youtu.be/qj97LjGcj1Y
I hooked up the PZEM-017 and got it working - well for awhile !
I was using the factory software and amp reading was off .
Then I thought yeah I had different shunt on there so I half read the instructions on how to do it.
The software had a Calibrate command - and oops that screwed it up !
Reading some other instructions said basically don't use the Calibrate - oops too late.
There was another spot in software to set the shunt.
Anyway might fixable by modbus ? But I just ordered another one.
Do you want the one I messed up ? Maybe you have the modbus skills to recalibrate it right.
If you want it I will send it to you - message me .
Larry
Quote from: ClassicCrazy on January 12, 2022, 05:33:37 PM
Graham -
Yeah I goofed up the link
Here it is - also I corrected it in the other post.
https://youtu.be/qj97LjGcj1Y
I hooked up the PZEM-017 and got it working - well for awhile !
I was using the factory software and amp reading was off .
Then I thought yeah I had different shunt on there so I half read the instructions on how to do it.
The software had a Calibrate command - and oops that screwed it up !
Reading some other instructions said basically don't use the Calibrate - oops too late.
There was another spot in software to set the shunt.
Anyway might fixable by modbus ? But I just ordered another one.
Do you want the one I messed up ? Maybe you have the modbus skills to recalibrate it right.
If you want it I will send it to you - message me .
Larry
sure, it would be fun to see if I can fix it, I'll send you a PM with my address etc.
If you're using telegraf as the mqtt subscriber for influxdb, you should be able to add the tasmota topic to the telegraf.conf file to get the telemetry data to grafana. I have a bunch of sonoff/tasmota switches, so I'll try it here to see if I can get that to work.
I started to update the wiki at https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack) to include influx/telegraf/grafana, you're more of an expert on grafana, let me know if I should add more details on the last section
Graham.
...
I added "tele/tasmotaPower_6839BD/SENSOR" to the telegraf.conf topics area and got the extra field keys for the sonoff P2. You can use the TelePeriod command in the tasmota console to change the publish rate for the sensor (default is 300s)
Quote from: Graham on January 13, 2022, 09:13:09 AM
I added "tele/tasmotaPower_6839BD/SENSOR" to the telegraf.conf topics area and got the extra field keys for the sonoff P2. You can use the TelePeriod command in the tasmota console to change the publish rate for the sensor (default is 300s)
Graham - I got my Sonoff POW added to mqtt_consumer via telegraph and seems to be working.
I haven't set it up on Grafana yet.
Thanks for that - I never would have imagined it would be that easy to do !
Larry
I was able to get the Sonoff data into Grafana . I am getting better at figuring out how to set up things in Grafana .
Larry
Here is the my latest Midnite Grafana dashboard flow with any additional improvements I have made since last time I posted it.
I just added shunt watts and also data from the Sonoff POW.
Larry
I didn't like the version of Grafana with auto-size fonts. I use v7.4.3 which has text size settings.
This is my ugly 1 of 5 Classics...
Quote from: Wizbandit on January 16, 2022, 12:34:58 PM
I didn't like the version of Grafana with auto-size fonts. I use v7.4.3 which has text size settings.
This is my ugly 1 of 5 Classics...
I am using Grafana 8.3.0
I use whatever docker version IOTStack installs . IOTStack makes it easy to automatically install all the needed programs like Portainer, Influxdb, Telegraph, Mosquitto, etc.
Pretty sure I can set whatever text size and fonts I want but havent' messed with that yet.
Right now I am learning how to have Grafana do the math to add up totals from both of my Classics and display it properly.
I had mistakes on other dashboards I shared and am now correcting them ( learning Grafana mostly by trial and error ).
Also here is my latest improved grafana dashboard
Quote from: ClassicCrazy on January 16, 2022, 01:49:21 PM
I am using Grafana 8.3.0
I use whatever docker version IOTStack installs . IOTStack makes it easy to automatically install all the needed programs like Portainer, Influxdb, Telegraph, Mosquitto, etc.
...
You can use IOTStack to generate a docker-compose.yml file to start with, then edit the file to suite your needs.
Once you have the docker-compose.yml setup the way you want it, you can use the docker commands to create and start the containers
you can also setup bash aliases https://sensorsiot.github.io/IOTstack/Docker-commands/ (https://sensorsiot.github.io/IOTstack/Docker-commands/).
If you want to pull a specific version of an image, you can specify any version that is defined in dockerHub https://hub.docker.com/r/grafana/grafana/tags?page=1&ordering=last_updated (https://hub.docker.com/r/grafana/grafana/tags?page=1&ordering=last_updated)
grafana:
container_name: grafana
image: grafana/grafana:7.4.3
...
note: if you run the IOTStack/build stack after you edit your docker-compose.yml file it will overwrite your edits...
Graham.
I was able to flash an ESP32 Devkit V1 board with Tasmota .
It wasn't documented very well but is supported and also supports the Peacefair PZEM-016 AC monitor .
I used a RS485 to TTL adapter board to connect to ESP32, selected the pin options for PZEM in Tasmota configuration menu and it worked !
( should work with esp8266 too)
The CT from PZEM is snapped around neutral wire from inverter to main breaker box. I have about 30 feel to cat 5 bringing signal from PZEM into house ( the PZEM board is out by breaker box)
Doesn't get much easier than that ( though it took me a few hours to find all the info I needed) .
Tasmota is great firmware to flash on ESP because drop down menu for every pin shows all available options for every type of sensor that may work .
I used this info for wiring https://community.home-assistant.io/t/how-to-connect-pzem-017-dc-with-nodemcu/160754/12
The PZEM is sending power back over cat 5 wire I am using and it powers the esp32.
I just need to configure the mqtt in Tasmota and should be able to bring it into Grafana by making the changes in telegraf.conf and setup database in influx as described in previous post.
Larry
Integrated PZEM energy monitor into Grafana
Quote from: ClassicCrazy on January 28, 2022, 06:57:53 PM
Integrated PZEM energy monitor into Grafana
Larry,
I recently updated the wiki in https://github.com/ClassicDIY/ClassicMQTT (https://github.com/ClassicDIY/ClassicMQTT) to add more details in the "Raspberry Pi setup using IOTStack" section.
I was wondering if we should add another section describing the integration of other devices like Tasmota.
Graham
Quote from: Graham on January 30, 2022, 08:21:30 AM
Quote from: ClassicCrazy on January 28, 2022, 06:57:53 PM
Integrated PZEM energy monitor into Grafana
Larry,
I recently updated the wiki in https://github.com/ClassicDIY/ClassicMQTT (https://github.com/ClassicDIY/ClassicMQTT) to add more details in the "Raspberry Pi setup using IOTStack" section.
I was wondering if we should add another section describing the integration of other devices like Tasmota.
Graham
Yeah why not ? Tasmota makes it amazingly easy to add things to mqtt and Grafana .
I can help with some screenshots or how to .
That is a nice graphic you made.
Larry
I actually built a prototype "Power Monitor" daughter board for the MNS COMBOX. It has 10 CT's and 4 AC voltage inputs. Not sure if it will make it to the final MNS DIY COMBOX as an option yet. I have been "tweaking" it and playing with it on my home solar system. It needs a simple user setup interface to be viable. Right now all the running code is "hard-coded" to fit my setup.
4 - Classic's (~ 13Kw Installed PV)
1 Sunny Island 6K & Hawkes Bay "selling" (barn ~ 3Kw PV)
1 Sunny Boy 3.8 TL (barn ~ 3Kw PV)
1 Aurora Power-one 3K (Barn ~ 2Kw PV)
Quote from: Wizbandit on January 31, 2022, 08:53:51 AM
I actually built a prototype "Power Monitor" daughter board for the MNS COMBOX. It has 10 CT's and 4 AC voltage inputs. ...
Nice Dashboard,
are you using MQTT or are you sending data directly to influxdb with the influxdb-python library ?
Graham.
I made three videos I put on youtube explaining how I wired up PZEM-016 and 017 energy monitors to Esp8266 or Esp32 . First video is on wiring, second video on setting up the mqtt on Tasmota flashed Esp , and third video is about getting the data on Grafana.
Here are the links
https://youtu.be/TbLLrlZ3deI
https://youtu.be/xKxGaerzNKM
https://youtu.be/BaCugQvhQ00
Larry
Quote from: Graham on January 31, 2022, 12:44:28 PM
Quote from: Wizbandit on January 31, 2022, 08:53:51 AM
I actually built a prototype "Power Monitor" daughter board for the MNS COMBOX. It has 10 CT's and 4 AC voltage inputs. ...
Nice Dashboard,
are you using MQTT or are you sending data directly to influxdb with the influxdb-python library ?
Graham.
Yes. I designed a circuit that uses two 8 channel ADC chips, a 3V stable reference, Chip Select decoder, and SPI interface on the Pi. I have a dozen projects going right now but when I can I plan on understanding what MQTT is. (think I looked it up once but can't remember) I only work part-time for Midnite so most of my work on the MNSCOMBOX and extras are done evenings and weekends.
Quote from: Wizbandit on February 03, 2022, 08:11:20 AM
...
Yes. I designed a circuit that uses two 8 channel ADC chips, a 3V stable reference, Chip Select decoder, and SPI interface on the Pi. I have a dozen projects going right now but when I can I plan on understanding what MQTT is. (think I looked it up once but can't remember) I only work part-time for Midnite so most of my work on the MNSCOMBOX and extras are done evenings and weekends.
MQTT is just a simple publisher/subscriber pattern that decouples the sender (publisher) with the receiver (subscriber). It would not make sense to use MQTT as the internal protocol if you are running influxdb/grafana on the MNSCombox. But you might want to consider including the option to publish to MQTT as the ClassicMQTT does for us DIYers so we can aggregate the data from the MNSCombox with other IOT devices like Larry's current sensor into subscribers like Node-Red, HomeAssistant or a custom Grafana dashboards.
Graham.
Graham
There were some changes to IOTstack or Docker which messed up running menu in IOTstack.
After a lot of time with help from paraphraser at IOTstack found that the problem was that networks has been changed.
In the end ( after I rebuilt my IOTstack from scratch) to fix the Classics mismatch all I had to do was change compose-overide.yml networks from iotstack_nw to networks default.
I sent a lot of messages on Slack but you can skip them - this was the conclusion .
classic_mqtt:
container_name: classic_mqtt
image: classicdiy/classicmqtt
restart: unless-stopped
environment:
- LOGLEVEL=DEBUG
- CLASSIC=192.168.3.22
- CLASSIC_PORT=502
- CLASSIC_NAME=CLASSIC
- MQTT_HOST=mosquitto
- MQTT_PORT=1883
- MQTT_ROOT=ClassicMQTT
- MQTT_USER=ClassicPublisher
- MQTT_PASS=ClassicPub123
networks:
- default
Quote from: ClassicCrazy on February 04, 2022, 11:11:26 AM
Graham
There were some changes to IOTstack or Docker which messed up running menu in IOTstack.
After a lot of time with help from paraphraser at IOTstack found that the problem was that networks has been changed.
In the end ( after I rebuilt my IOTstack from scratch) to fix the Classics mismatch all I had to do was change compose-overide.yml networks from iotstack_nw to networks default.
I sent a lot of messages on Slack but you can skip them - this was the conclusion .
classic_mqtt:
container_name: classic_mqtt
image: classicdiy/classicmqtt
restart: unless-stopped
environment:
- LOGLEVEL=DEBUG
- CLASSIC=192.168.3.22
- CLASSIC_PORT=502
- CLASSIC_NAME=CLASSIC
- MQTT_HOST=mosquitto
- MQTT_PORT=1883
- MQTT_ROOT=ClassicMQTT
- MQTT_USER=ClassicPublisher
- MQTT_PASS=ClassicPub123
networks:
- default
Thanks for finding that error Larry, I updated the compose-override.yml file on the github page https://github.com/ClassicDIY/ClassicMQTT (https://github.com/ClassicDIY/ClassicMQTT).
Also updated the wiki using the "old" IOTStack menu since it didn't give me that docker version warning. (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack))
I tested it on a fresh Raspberry Pi 3 32Bit OS
Graham
Quote from: Graham on February 05, 2022, 11:04:07 AM
Quote from: ClassicCrazy on February 04, 2022, 11:11:26 AM
Thanks for finding that error Larry, I updated the compose-override.yml file on the github page https://github.com/ClassicDIY/ClassicMQTT (https://github.com/ClassicDIY/ClassicMQTT).
Also updated the wiki using the "old" IOTStack menu since it didn't give me that docker version warning. (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack (https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack))
I tested it on a fresh Raspberry Pi 3 32Bit OS
Graham
There is a section on IOTstack that shows how to change between menus pretty easily.
I didn't see the usual docker out of date warning when I switched back to new menu.
https://sensorsiot.github.io/IOTstack/Getting-Started/
Larry