I want to try Jose's node red flow from this post.
http://midniteftp.com/forum/index.php?topic=4854.msg52373#msg52373
So I thought I would start out with fresh Raspberry Pi OS install. The newest one is called Bullseye replacing Buster.
First thing I ran into was I was running headless - pi connected via wireless and then using my windows computer VNC Viewer to get into the Pi's desktop.
The desktop was not showing up full monitor.
I found this youtube video which showed how to fix this issue ( took me awhile to find this solution)
https://youtu.be/-7rlLJmNmfA
Node Red came already installed in the Bullseye version I had used . But when I ran it there were some version errors shown.
I found this after some searching https://nodered.org/docs/getting-started/raspberrypi
From the link I ran this in a terminal window
bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
That seems to have fixed the version issues that I was seeing before.
I imported Jose's flow
I had to install node-red-contrib-modbus to pallete to get the modbus features for node-red
Also I had to install node-red-contrib-influxdb because Jose's sketch also uses that.
I need to install influxdb and grafana so decided to try to do it the easy way with IotStack project that uses Docker . I will find out if that is easy or not !
from this page https://sensorsiot.github.io/IOTstack/Getting-Started/
Follow New Installation on above link
Install all went fine for me. Restart
CD into IotStack then run ./menu.sh
Prompted to do some updates - did those all is well
Found out some of the updates for IOTstack not needed since they were already up to date but got prompted anyway.
This one sort of messed up my pi
"â", Docker version is currently 20.10.5+dfsg1 which is less than 18.2.0 â",
â", consider upgrading or you may experience issues. You will not be prompted â",
â", again. You can manually upgrade by typing: â",
â", sudo apt upgrade docker docker-compose â","
â",
So don't do it , for some reason it made my router choose a different IP address for the pi - took me awhile to figure out.
A few bugs with the IOTstack . I am using it to install Grafana, Influx, Mosquitto and Portainer
Back to Node Red
in command prompt node-red-start
Go to Browser http://127.0.0.1:1880/
to get Node Red webpage
Imported Jose flow again ( it didn't save from last time for some reason)
Gave up on figuring out how to make everything work in Node Red
Went on to install Create the ClassicMQTT container using IOTstack:
from this link https://github.com/ClassicDIY/ClassicMQTT/wiki/3.1-Raspberry-Pi-setup-using-IOTStack
which also tells how to install IOTstack
but since I already have it from above - I just wanted to install the MQTT for Classic.
Even though Graham has detailed instructions and screenshots - I messed up following a simple instruction and took a couple hours to figure out what I did wrong.
which was instead of "nano compose-override.yml" like Graham says
I was doing " nano docker-compose.yml" which of course was wrong and would erase every time I tried to run it which I tried way too many times !
Anyway once I did it right it started right up and was able to use Graham's Classic Monitoring app on Android in MQTT mode.
Next I will see if I can adapt Jose's node red project to get the data from MQTT instead of from modbus.
That is enough for me for tonight !
Larry, you can use Telegraph instead of node-red to subscribe to the ClassicMQTT messages and update influxDB directly
-Include Telegraph in the IOTStack build stack
- Grab the attached telegraf.conf, update the topics, username and password for the MQTT consumer (if your broker is not in a sibling container, also update the servers)
- ftp the telegraf.conf to ~/IOTstack/volumes/telegraf on your RPi
This will populate a database called mqtt_solar that you can setup as a dataSource in grafana
- Setup the mqtt_solar datasource in grafana
- import the attached grafana_dashboard.json
Graham.
thanks Graham - I will give it a try
Quote from: Graham on December 01, 2021, 10:03:42 AM
Larry, you can use Telegraph instead of node-red to subscribe to the ClassicMQTT messages and update influxDB directly
-Include Telegraph in the IOTStack build stack
- Grab the attached telegraf.conf, update the topics, username and password for the MQTT consumer (if your broker is not in a sibling container, also update the servers)
- ftp the telegraf.conf to ~/IOTstack/volumes/telegraf on your RPi
This will populate a database called mqtt_solar that you can setup as a dataSource in grafana
- Setup the mqtt_solar datasource in grafana
- import the attached grafana_dashboard.json
Graham.
I think I have the telegraf .conf file installed
But setting up the Grafana datasource I get bad gateway error .
I haven't been able to figure this out yet .
Screenshots show my grafana
you need to set your URL to http://influxdb:8086 in the datasource
(assuming your container name for influxdb is influxdb)
Thanks Graham - putting in Influxdb solved that gateway issue.
Now I get Influxdb database not found error.
Seems like IOTstack did some update and messed itself or everything else up.
I am going to wipe it all out tonight and start over.
Larry
The telegraf.conf file is using the influxdb user/passwd that IOTStack creates by default, check the logs for telegraf (quick actions icon in portainer) to see if you're getting errors
Quote from: Graham on December 05, 2021, 09:46:23 AM
The telegraf.conf file is using the influxdb user/passwd that IOTStack creates by default, check the logs for telegraf (quick actions icon in portainer) to see if you're getting errors
I wiped SD card and reloaded Raspberry Pi Bullseye and also IOTstack .
The last IOTStack update I did really messed things up. Or could be I messed things up.
I think there are lots of updates on both Bullseye and IOTstack being rolled out quickly so maybe second time fixed some previous errors.
And I found out I can make errors just as well in the daylight as in middle of night.
Latest one was I messed up the Telegraph .conf but spotted it in errors ( where you told me to check).
Anyway happy to report that now it is all working with your imported Graphana files.
Thanks for the help - I always learn a lot.
Here is screenshot ( super dark cold overcast day here so generator is running )
Larry
looks good,
The next step would be to convert the state of charge codes to English text. this is more difficult than you would expect in grafana.
You need to add the mariadb (mysql) container, create a lookup table in sql with the SOC codes and associated English text, then add a join statement to the SOC to do the lookup in grafana
something similar to this:
https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/ (https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/)
Graham.
Quote from: Graham on December 06, 2021, 08:04:04 AM
looks good,
The next step would be to convert the state of charge codes to English text. this is more difficult than you would expect in grafana.
You need to add the mariadb (mysql) container, create a lookup table in sql with the SOC codes and associated English text, then add a join statement to the SOC to do the lookup in grafana
something similar to this:
https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/ (https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/)
Graham.
Figures nothing is too simple !
I have been looking a little bit under the covers at how you set the graphics up.
But have not spent too much time on it yet.
Larry
I figured out how to change the temperatures from Celsius to F
Just had to edit temp block and change this line
select BatTemperature * 0.018 + 32 from mqtt_consumer
easy once I spent an hour figuring out the right place to put this simple fix in .
Quote from: Graham on December 06, 2021, 08:04:04 AM
looks good,
The next step would be to convert the state of charge codes to English text. this is more difficult than you would expect in grafana.
You need to add the mariadb (mysql) container, create a lookup table in sql with the SOC codes and associated English text, then add a join statement to the SOC to do the lookup in grafana
something similar to this:
https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/ (https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/)
Graham.
Graham
How do I get a list of the MQTT identifiers ?
such as PV Current , BatTemp , etc like in "select PVCurrent from mqtt_consumer"
I was thinking about adding my second Classic into config but don't know how Grafana would know which PVCurrent to get since they seem to be both identified the same way once in Telegraf ?
I may tackle the charge state sometime as you mentioned above.
Thanks for getting me started on this - works great. I now monitor the Grafana page that is running on the Pi , but look at it on my Windows computer .
I will have to do one of the intro to Grafana tutorials so I fill in the many blanks of what I don't know about it
Larry
Quote from: ClassicCrazy on December 09, 2021, 07:23:51 PM
Graham
How do I get a list of the MQTT identifiers ?
such as PV Current , BatTemp , etc like in "select PVCurrent from mqtt_consumer"
I was thinking about adding my second Classic into config but don't know how Grafana would know which PVCurrent to get since they seem to be both identified the same way once in Telegraf ?
I may tackle the charge state sometime as you mentioned above.
Thanks for getting me started on this - works great. I now monitor the Grafana page that is running on the Pi , but look at it on my Windows computer .
I will have to do one of the intro to Grafana tutorials so I fill in the many blanks of what I don't know about it
Larry
There is a payload example for "ClassicMQTT/classic/stat/readings" in the classicDIY/ClassicMQTT github wiki: https://github.com/ClassicDIY/ClassicMQTT/wiki/5.-MQTT-topics-and-message-structure (https://github.com/ClassicDIY/ClassicMQTT/wiki/5.-MQTT-topics-and-message-structure)
If you want to monitor a second classic:
1. Update your compose-override.yml file in IOTstack to include your second classic (see attached example)
2. Edit your telegraf.conf file using: sudo nano ~/IOTStackvolumes/telegraf/telegraf.conf
3. Add the topic for your second classic, then add topic_tag = "topic" (see attached example).
4. run ~/IOTStack/menu.sh,
5. Build Stack, the Enter to merge your compose-override.yml into the docker-compose.yml
6. Docker Commands, Start stack to run the containers, then exit IOTStack
7. You should now see classic_mqtt2 in portainer, check the logs to make sure it's connecting to your second classic
8. Restart the telegraf container
9. In Grafana, you now have to include a "where" clause in the select statements for each panel
ex: select BatVoltage FROM "mqtt_consumer" WHERE ("topic" = 'ClassicMQTT/MyShed/stat/readings')
You can use the grafana datasource explorer to query the series in mqtt_consumer, go to the influxdb datasource and press the explore button at the bottom of the page.
You can also use the influxdb container console, run the influx command (see attached screen shot)
Graham.
Graham ,
I was able to get it all set up in Grafana for two Classics now.
Of course I made a few mistakes first but that really helps me learn !
I used Portainer to get into the container consoles and then your example helped me figure out how to go from there.
It was super overcast today so nothing exciting on the screenshot but I did get it running. Also figuring out now how to do things in Grafana
thanks again for the help.
(ooops forgot to save dashboard so have to do it all over again ! more practice )
Larry
Quote from: Graham on December 06, 2021, 08:04:04 AM
looks good,
The next step would be to convert the state of charge codes to English text. this is more difficult than you would expect in grafana.
You need to add the mariadb (mysql) container, create a lookup table in sql with the SOC codes and associated English text, then add a join statement to the SOC to do the lookup in grafana
something similar to this:
https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/ (https://grafana.com/blog/2019/07/17/ask-us-anything-how-to-alias-dashboard-variables-in-grafana-in-sql/)
Graham.
I may have found easier way to do the State of Charge code to text.
There is a Value Mapping section .
https://grafana.com/docs/grafana/latest/panels/value-mappings/
Guess I will find out next time the sun comes out if it works or not.
Screen shot shows what I did .
attached is my latest version of Grafana Dashboard supporting two Classics and I jazzed it up with colors.
Larry
State of Charge working .
I also added totals between the two Classics.
Larry
Quote from: ClassicCrazy on December 10, 2021, 07:57:03 PM
I may have found easier way to do the State of Charge code to text.
There is a Value Mapping section .
https://grafana.com/docs/grafana/latest/panels/value-mappings/
Guess I will find out next time the sun comes out if it works or not.
Screen shot shows what I did .
attached is my latest version of Grafana Dashboard supporting two Classics and I jazzed it up with colors.
Larry
That's much easier than the SQL approach... thanks for finding that.
I made some progress with Node Red - figured out how to get it to read the MQTT data .
Easy I suppose once you know what you are doing !
Thanks to Grahams past instructions on how to use MQTT.fx program - that is helping me get some understanding of MQTT stuff.
Next I will work on trying to make Node Red use some of the data to control something.
I attached the flow I have so far which will show how to setup the mqtt input .
Larry
I worked on getting Node Red to take Aux data, then change the true / false to a number value.
Then it is supposed to go to Influxdb but I haven't got the influx databases configured right yet.
Here is what I have used for references to get this far
Very good video where he takes time to explain each step in detail
https://youtu.be/ffg3_1AgtyA
Also Paraphraser helps some though for me hard to follow along sometimes until I read it over and over
https://gist.github.com/Paraphraser/c9db25d131dd4c09848ffb353b69038f
This is the function code I used to change true false to number value - I found this while searching for way to do it so thanks whoever wrote it cause I forgot already
node.warn(msg.payload)
msg.original = msg.payload;
if (msg.payload === "true" || msg.payload === true) {
msg.payload = 111;
} else {
msg.payload = 999;
}
return msg;
Larry