Node-Red, ModBus, InfluxDB, Grafana on Raspberry Pi - Solar System Monitoring

Started by Steve_S, June 08, 2019, 05:31:42 PM

Previous topic - Next topic

dgd

I have had a while to read up on these software packages for the Pi3. The modbus library is good although I would prefer its use via a direct serial port connection to the Classic rather then some Ethernet to serial conversion. Doesn’t seem to onerous to implement once the various timings are sorted out.
The InfluxDB and Grafana look interesting but I’m left with the probably incorrect impression that they are an awful lot of overhead for dealing with just a relatively few data from the Classic. Maybe I’m just too old fashioned but using a data structure managed by some simple C code to extract and store Classic data over time OR for a real time metered display looks like an easy approach. Grafana is just a graphics package, looks like it can make some whizzy displays but I’d like to see some better or bettered dashboard metering (of have I missed this).
Anyway, I look forward to seeing how yous progress (CC and Steve) as I want to learn.
In meantime I will tidy up my Pi3 c code, document and post my simple Classic monitoring tool for Pi via serial port.
Dgd
Classic 250, 150,  20 140w, 6 250w PVs, 2Kw turbine, MN ac Clipper, Epanel/MNdc, Trace SW3024E (1997), Century 1050Ah 24V FLA (1999). Arduino power monitoring and web server.  Off grid since 4/2000
West Auckland, New Zealand

ClassicCrazy

Not sure what you mean dgd by a lot of overhead - the Pi Zero W only costs about $10 and uses about 1 watt or less. I don't know though if it has enough processing power to run the Grafana and all though . Even though I have a pi zero W I have been so far using Pi 3 B+ to experiment with the node red - though I also often just use my windows computer with node red too since it is usually always running in the daytime anyway.
I haven't tried hooking the pi up to serial on Classic - well I did try but I figured out that since I have the Classics in Follow Me I can't get serial data . Maybe I will try putting both the Classics as independent controllers and see if I can get the serial data going - like you say should work a lot better than modbus IP .

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

dgd

CC, sorry I didn’t mean power usage or money investment in Pi3, instead I meant cpu cycles as I have found the Pi tends to run out of grunt which can often lead to lock ups or weird results. The overhead of a DB inc it’s language interpreter then a well featured graphics generator all running real time data extraction and display, I think will push the Pi3. Maybe Im wrong but we will see when I get time to install everything.
With my Pi3 I was thinking of using it dedicated to classic data extraction and offering that data as some web pages, but even with minimum web server code the Pi3 resource is getting stretched.
Dgd
Classic 250, 150,  20 140w, 6 250w PVs, 2Kw turbine, MN ac Clipper, Epanel/MNdc, Trace SW3024E (1997), Century 1050Ah 24V FLA (1999). Arduino power monitoring and web server.  Off grid since 4/2000
West Auckland, New Zealand

Steve_S

At this point, I have Node-Red running, pulling Modbus via TCPIP with no issue, the Modbus lib I am using can also do serial but why If you can get it via IP.  With NodeRed, InfluxDB & Grafana running on my Pi3, it sees spikes of 12% when writing to Influx and 20% if I am working software on my browser.  I do not use the browser in the Pi for more than just settings, I use my lappie to connect to the Pi.  The Pi can handle this well enough as a "server" but don't think you'll be browsing or doing much more on it with all of this running.

The writes are but a blink literally and don't create IO bottlenecks BUT I am running off an SSD drive, which likely has a positive effect on that.

My Nodered flow is simple enough, it's only pulling the data I want, setting it up and dumping it to Influx.  The basic Dash (nodered) was just to see a quick & dirty "it is working" screen.   I was originally thinking of writing a program in C# which was cross platform BUT with RasPI and all these goodies that simplify it and being all FREE it's a no brainer to go this route.  There is a CATCH for me though.  My Samlex gear can also talk Modbus BUT I had to sign an NDA to get their data & specs, so I won;'t be able to share that part for now, till things are sorted.

SOMETHING FOR PI FOLKS !
As mentioned, I went with an SSD drive on my Pi and it is terrific to use it... I followed instructions from Andreas Speiss on YouTube and all went according to plan BUT I just came across this which solves a LOT of quirks & foibles.

Geekworm Raspberry Pi 3 B+/3B SATA HDD/SSD Storage Expansion Board, X820 V3.0 USB 3.0 Mobile Hard Disk Module for 2.5 Inch SATA HDD/SSD/Raspberry Pi 3 Model B+ (B Plus)/3 B/ROCK64/Tinker Board https://www.amazon.com/Geekworm-Raspberry-Storage-Expansion-Compatible/dp/B073CGQD9C/ref=sr_1_19?keywords=Orange+Pi&qid=1560892223&s=gateway&sr=8-19   and it is ONLY $28   Plus Geekworm has cases with fans and expansion modules to house your Pi & attached kit, all very reasonably priced and most affordable.

SIDE NOTE: There are Banana Pi's & Orange Pi's which have faster CPU's and up to 4GB Ram onboard which are 100% compatible with Raspberry, same form factor and compatible with the extra goodies.  Prices are very very close to RasPi's as well, so not a big leap if that's a point of contention.

Raster

Great. I am trying to do the same thing for my (future) solar system: Rpi + Node-Red + InfluxDB  + Grafana. Arduinos will run various sensors: temp, voltage, and GPS (for  time). The IDE for the Arduino runs on the Rpi. Don't know if the RPi has enough horsepower, but so far it does. The Arduino is powered and communicates on a USB port on the RPi. So far, I have the Node-Red dashboard running showing some sample sensors from the Arduino. Haven't started on InfluxDB and Grafana. What is nice is that the dashboard is viewable by any browser. All this software is free, and RPis and Arduinos are very inexpensive and low power (3-4 watts at present). Definitely will look up Andreas on YT.

Vic

Hello Steve_S,

Am tardy in thanking you for generously sharing your work.  At this point,  it is way over my head,   but hope to delve into the R pi more,   if we ever catch up on too many other projects.

Am also following CC Larry,   and dgd's projects,   Thanks,   all,   Vic
Off Grid - Sys 1: 2ea SW+ 5548, Surrette 4KS25 1280 AH, 5.25 KW PV, Classic 150,WB, Beta Barcelona, Beta KID
Sys 2: SW+ 5548s, 4KS25s, 5.88 KW PV, 2 ea. Classic 150, WB, HB CC-needs remote Monitoring/Control, site=remote.
 MN Bkrs/Bxs/Combiners. Thanks MN for Great Products/Svc/Support&This Forum!!

Steve_S

Quick update / response...
I'm still tinkering with this BUT it is low priority at the moment, that miserable time of year is coming and too much to do in the meantime.   
I've received my USR RS232/RS485 -> TCPIP (for Modbus) converter which will interface with my Samlex EVO4024 Inverter so I can extract data via Modbus...   Very interesting little kit, looking forward to tinkering with that, I chose that route instead of a "hat" for the RasPi as I wanted a tad more flexibility.

Here is a TWIST:   As my Powerhouse is in a separate building 75' away, I've decided to run a 24VDC->12VDC converter (direct from my battery bank) to a pair of Cigarette Lighter Sockets which will, hold some ANKER 12V-USB Adapters which will power the RasPi (5v/2.5A) The USR (5v/1a) and my Router which is 12V/1.0A while leaving the Satellite Modem in the can on a switch.  This will allow the DC Devices to operate even when the inverter is off  and without going through the losses of converting up & down.

Influx DB:   I have gotten it to store the Classic Data quite nicely but I'm going t tweak that out to give me better data without the clutter...  so I guess I'll be doing some SQL processes for that....   

Not really much more progress than what has been posted here at this point... but I will be working on it once the next 4 projects are completed.

ClassicCrazy

Sounds like a good plan to use the 24 to 12 converter to power your stuff directly like that . Pretty much how I do my DC loads. I also use the small $1  buck converters to drop 12v to  3v or whatever other voltage I need since they are adjustable . I have eliminated the batteries on weather stations, etc by just soldering a couple wires onto the battery connectors - one of those buck converters will power a lot of devices . I seem to collect lots  wireless temperature gauge at garage sales when they are less than a buck - then plug them all in for the fun of it. Fun ! https://www.ebay.com/itm/1-5PCS-LM2596S-3A-DC-DC-Power-Supply-Buck-Converter-Adjustable-Step-Down-Module/262770509484?hash=item3d2e57c6ac:m:mkmg3y3Hc7Nb2pOqFK7-7ZQ

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Steve_S

Just a quick note on this thread.  It's NOT DEAD !   I've been terribly busy with construction, moving & finishing a build as well as a few other things, I haven't forgotten about this and it will proceed once things settle down a tad.

An Evolution is happening to the concept though as I don't like to do things more than once, ok twice on occasion...   In my quest for monitoring & control over power usage along with a need t automate a few things to simplify life I'm expanding the project in a few ways. 

I've decided that part of the implementation will also handle / control IoT enabled "Smart Plugs" which have energy use monitoring functions as well as being able t shutdown devices and even schedule them for use.  To that end, I've spent a couple of weeks poking around, reading reviews and looking at development platform capabilities for Node-Red, Raspi and the goodies I'm using.  After all of that, I decided to use Samsung's "Smartthings" â,,¢ hardware for the IoT devices which also connected with "Home Assistant" and many other goodies and can be addressed within NodeRed.   Additionally, the system will have a Weather Station connected into the system as things progress.   Nice part is that several pieces should work with InfluxDB & Grafana and I think a nice respectable interface will be possible.

REFERENCE INFO:
Samsung Smartthings :  https://www.smartthings.com/   *
*  Note the "New" power plug has energy use monitoring while the old one does not.  Cheapest through Amazon BUT be aware of the different plugs V1 or V2.


ClassicCrazy

I am sure a lot of us understand the too busy with projects !
I played a bit with Node Red - that is how I first used it - with a Sonoff outlet.
I used this turorial. https://randomnerdtutorials.com/sonoff-basic-switch-esp-easy-firmware-node-red/
You will find that installing EspEasy gives a lot of options - or just put it into any esp8266 or esp12 and it supports all kinds of devices and the mqtt sites ,etc. I started on trying to install Mosquito MQTT on Raspberry Pi so I could control stuff more locally but ran into springtime so put the winter projects to the side for now.
There are newer EspEasy Mega releases than shown in the tutorial .
https://github.com/letscontrolit/ESPEasy/releases
I have $3 esp8266 outside with a few temp sensors - been outside in a box for a couple  years now and it just chugs along and sending out wifi continuously - amazingly stable .

Will look forward in the future to your projects.

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Steve_S

Just a quicky note.

I have been occupied with other priorities but will be getting into this project soon (winter is insisting) but there is a New Twist to it as well.   I have purchased a 24V/400AH LFP pack for my system but it came with a crappy BMS which is on it's way into a waste bin !   I am replacing that BMS with a unit from Chargery, an BMS8T which has many features, functions and capabilities, INCLUDING RS232 Communications to PC, so of course that will become a part of the NodeRed project

Attached is the very simple Chargery RS-232 interfacing doc (PDF)

REF:  http://chargery.com/BMS8T.asp

Westbranch

KID FW1811 560W >C&D 24V 900Ah AGM
CL150 29032 FW V.2126-NW2097-GP2133 175A E-Panel WBjr, 3Px4s 140W > 24V 900Ah AGM,
2 Cisco WRT54GL i/c DD-WRT Rtr, NetGr DS104Hub
Cotek ST1500 Inv  want a 24V  ROSIE Inverter
OmniCharge3024  Eu1/2/3000iGens
West Chilcotin 1680+W to come

Steve_S

There are absolutely no markings whatsoever on there for the one supplied by ShunBin in their pack.   Actually looks like my 3 year old soldered it together to be honest.

The only info I got was by E-Mail after bugging them via amazon communications is below:
---  Answer 1
Hello, the charging voltage is: 28.8V-29.2V ,The continuous discharge current of BMS is 300A,BMS can be suitable for 100A,Lifepo4 is easy to use, what recommendations parameter do you need?
---- Answer 2
Charging temperature : 0~45 â,,ƒ Discharge temperature : -20 ~ 60 â,,ƒ Storage temperature : -20 ~ 45 â,,ƒ Lo Voltage Cutoff 20V,There is no Lo Temp cutoff ,Only high temperature cutoff, we don't have a manual yet,The charger output current is 50A, BMS (Battery Management System): With built-in BMS  for over-current, over-discharge, over-charge, short-circuit, adverse-polarity connection, balancing functions
------------------

Extended discussion on the BMS here and at the top is a link to the Actual ShunBin pack thread as well...
https://diysolarforum.com/threads/need-smart-bms-as-replacement-for-shunbin-pack.2046/

ClassicCrazy

That looks like a nice BMS Steve - good to know there is support for it . Will look forward to hearing how well it works when you get it all installed.  I still have the same question though about charging the lithium - what will you use to go from absorb to float - voltage alone or will you also be using ending amps ? I will wait until you get some real world experience there to see what works best .

Larry
system 1
Classic 150 , 5s3p  Kyocera 135watt , 12s Soneil 2v 540amp lead crystal for 24v pack , Outback 3524 inverter
system 2
 5s 135w Kyocero , 3s3p 270w Kyocera  to Classic 150 ,   8s Kyocera 225w to Hawkes Bay Jakiper 48v 15kwh LiFePO4 , Outback VFX 3648 inverter
system 3
KID / Brat portable

Steve_S

I'll be using these settig sin the Classic & Samlex.  I could wire in the relay from the BMS to shut off charging but I don't think that's nescessary.  The real buggaboo will be keeping the FLA's topped off.

Bulk Voltage: 28V - 29.2V
- Absorption Voltage *: 28V - 29.2V Absorption Time 0-15 min
- Float Voltage *: 26.6V - 27.6V
- Equalize *: OFF or as short time possible @ float voltage
VOLTAGE CUTOFFS Low Voltage: 22V - 24V High Voltage: 29.2V
* not necessary
I'm making the "assumption" from my collected research, that by using a lower rate @ 28V to charge vs 29V can help mitigate some heating potential and extend life cycles (very generalised) but there seems to be some extended debate on that, so I choose to "Play it Safe".