CCDM - Charge Controller Data Manager

[ClassicCrazy]

Morning folks!

Mike -> great job getting your system up and running! I'm glad the new guide seems to be better.
I will add a note about rebooting ad I had to do the same during one install trial.

Larry -> I can indeed add notes about the settings you found and am going to add a note about the python --version (vs python3) command as per your findings as well.
In terms of Follow Me scenarios, CCDM would only be looking at what ever is reported by controller 1, I have not finished up the code for multiple controllers.
Now, I'm not 100% familiar with how follow me works internally, if controller one reports the summation of all following controllers? I don't know but will read up on it.
I guess in the case of a shared whizbang you should be seeing the current of both controllers.......anyhow I'll have to give that scenario some thought.
And I'll have to monitor peak whizbang current as well since this would be additive of the two controllers in your case.
My first step will be to get multiple controller code finished and then handle follow me scenarios and do an auto detect of a set of controllers in follow me mode.....



I'm not sure if I'll get to any work on it done today, then tomorrow we head to our cabin for a week and will be off line for the most part.
We have internet there but I tend to not look at the phone much while in the bush and have no pc up there.
So I'll check back in early in the new year and get some of these updates completed.

Guys, thanks again for the feedback and sticking with it......
Take care and Happy New Year to all:

Brent

Brent - don't worry about the Follow Me. The second ( or multiple ) controllers simply follow the Absorb and Float from the master instead of making their own decision on what to do.  They also can share the battery temperature sensor. Otherwise there is no difference. I have mine wired up to share the single shunt - not sure how others do it. So the shunt Whizbang on mine shows the total of two controllers going into the battery but I can see each controller individually output as normal.
Enjoy your New Years get away out in nature away from all the tech stuff !
Larry

[Fort Wisers]

Thanks, Brent
You gotta understand this is from a guy that took two hours to get the LED to blink on an Ardunio.

While I haven't had a lot of time to play with CCDM, I will pass on a few observations.

1. The graphics (on my laptop) are covered by the data panels (widgets?) The battery icon is barely visible.
2. On the python window, I see a lot of errors flying by. They go so fast, I can't read them. It says something like:error reading reg4106-4108 block of charge controller.

There are maybe ten lines like this with different errors on different blocks.

While Mr Sun hasn't shown his face for over two weeks, having peaking reading values is well worth the price of admission.

On my wish list..

there is so much text, How about a panel, all it system status or something like that with some simple idiot lights for us idiots. Like 'charging' absorb, float and so on.

I guess it's the geek in me that can sit and watch numbers tumble around on a computer screen all day.

You have an amazing piece of software... Enjoy the down time at the cabin.

mike

Hey Mike!
Give yourself some credit my man, many people don't even know what an arduino is!
To answer your questions:
1. The graphics are auto resized based upon your devices native screen resolution, but there is a limit to this, what is the display resolution of your device?
You can find this in Win10 by right clicking anywhere on your desktop that does not have an icon and selecting "display settings".
If you're down in the 1024x768 range you will experience this overlap of widgets and images (with the widgets taking priority since they're actually displaying data where-as the images are "just for show"), I did some testing on screen resolutions this low and to resize the images small enough to avoid the overlapping, while also leaving the widget text large enough to be of any use (at least for my old eyes) became a bit of a challenge, so this was a bit of a compromised solution.
There is, however, a way to over-ride this.......we can explore this if all else fails.
Your best option, if possible depending on your device, is to try and increase your screen resolution, I'm currently running 1920x1080 and there is zero overlap of any screen elements.
You will have to restart CCDM each time you change the screen resolution as CCDM only auto sizes upon starting.

2. This sounds like a possible networking issue to me. Since your Charge controller is set at a 10.x.x.x address I'm thinking it is the default address set by Midnite and perhaps not set to a value best suited to your home network?
On your Win 10 machine, go to the start menu search bar, type in "network", select the network status app, then when the app launches, select "status" then select the "properties" button. Once in properties scroll down and find "IPv4 address", this will be the network and device address of your PC (wrapped up into one address, probably something like 192.168.1.x), let me know what this address is by PMing it to me, DO NOT POST IT ON THIS THREAD (it's likely not a public address, but if it is you don't want the world knowing it). It can tell us a fair bit about your network....from here we can setup the addressing on your classic correctly (if it's indeed not done yet). I'll try to sneak in an answer tomorrow morning before we leave depending on what you find and see if that helps with this issue.
A last check as well, what firmware is your classic running?


Now onto your wish list comment about a simple screen. I have a number of reasons I started this project. One was a friend of mine has a charge controller from a different manufacturer, his CC simply has a red, yellow and green LED......he never liked having such limited data from his CC. So I told him I was going to develop an application to get him more data (his CC has modbus - TCP/IP capability like the Classics). I started with a Midnite Classic because that's what we run at our cabin (and wanted to see our CC's data on our tv and beyond) but will migrate to include his CC type. So I'm a little hesitant to create what I was actually coding to solve. That said, we have lots of data on CCDM, so could we find a way to add a summary for those that don't really care to always sift through the details? Sure we could! Let me put some thinking into this as well over the holidays because you're correct, sometimes it's just nice to know, at a glance "is my system ok or not". I think we can find a nice balance and come up with a great summary screen, perhaps we simply add these simple summary elements to what is already there? Or perhaps it's best we just do a brand new "overall status" tab with less of the nitty gritty details? We will think of something.

Thanks for the feedback!


Take care, have a good one:
Brent

[qrper]

Brent,

Answers for some of you questions.

The display on my laptop is its default of 1366 x 768

I can't believe we have a day of bright sun, so CCDM is going wild with numbers. That's kinda cool looking at all the parameters change by the second. However, I can't tell if the Classic went to absorb or not unless I go the MISC screen and see if the absorb reached today is yes or no.

So, on the main page, there is no way to tell if the absorb voltage has been obtained, unless you know what the absorb voltage is. In my case, it's 58.8 V.
Now that being said, if a cloud goes by, the voltage falls, and I'm no longer in absorb, and there is no way to tell if you are or aren't in absorb mode

A simple indicator or two would certainly clear this up. And, I understand your thinking about get away from the idiot lights in the first place. That does make sense.

I'm enjoying the snot of of CCDM, so much that I'm thinking of getting a large monitor and a cheap laptop for the power room in the garage. I'd let it run all the time.

mike

[ClassicCrazy]

Brent,

Answers for some of you questions.

The display on my laptop is its default of 1366 x 768

I can't believe we have a day of bright sun, so CCDM is going wild with numbers. That's kinda cool looking at all the parameters change by the second. However, I can't tell if the Classic went to absorb or not unless I go the MISC screen and see if the absorb reached today is yes or no.

So, on the main page, there is no way to tell if the absorb voltage has been obtained, unless you know what the absorb voltage is. In my case, it's 58.8 V.
Now that being said, if a cloud goes by, the voltage falls, and I'm no longer in absorb, and there is no way to tell if you are or aren't in absorb mode

A simple indicator or two would certainly clear this up. And, I understand your thinking about get away from the idiot lights in the first place. That does make sense.

I'm enjoying the snot of of CCDM, so much that I'm thinking of getting a large monitor and a cheap laptop for the power room in the garage. I'd let it run all the time.

mike

Maybe get a raspberry pi - it will run on less power .
I might try to set up the CCDM on my pi just for the fun of it.

Larry

[qrper]

Brent,

Answers for some of you questions.

The display on my laptop is its default of 1366 x 768

I can't believe we have a day of bright sun, so CCDM is going wild with numbers. That's kinda cool looking at all the parameters change by the second. However, I can't tell if the Classic went to absorb or not unless I go the MISC screen and see if the absorb reached today is yes or no.

So, on the main page, there is no way to tell if the absorb voltage has been obtained, unless you know what the absorb voltage is. In my case, it's 58.8 V.
Now that being said, if a cloud goes by, the voltage falls, and I'm no longer in absorb, and there is no way to tell if you are or aren't in absorb mode

A simple indicator or two would certainly clear this up. And, I understand your thinking about get away from the idiot lights in the first place. That does make sense.

I'm enjoying the snot of of CCDM, so much that I'm thinking of getting a large monitor and a cheap laptop for the power room in the garage. I'd let it run all the time.

mike

Maybe get a raspberry pi - it will run on less power .
I might try to set up the CCDM on my pi just for the fun of it.

Larry

I looked at them last night. It appears they are quite robust for such a small footprint. More 'computer' than an Ardunio. This will require much more reading on my part. However that being said, it would appear a $40 slice of pi and a monitor would be all I'd need for a dedicated Classic monitor.
Shooting from the hip, and I admit I haven't looked too deep, but I'm not sure how one gets the CCDM software into the raspberry pi? Load it on a SIM card? External HD of some sort? More questions than I have answers to.

Mike

[ClassicCrazy]

Brent,

Answers for some of you questions.

The display on my laptop is its default of 1366 x 768

I can't believe we have a day of bright sun, so CCDM is going wild with numbers. That's kinda cool looking at all the parameters change by the second. However, I can't tell if the Classic went to absorb or not unless I go the MISC screen and see if the absorb reached today is yes or no.

So, on the main page, there is no way to tell if the absorb voltage has been obtained, unless you know what the absorb voltage is. In my case, it's 58.8 V.
Now that being said, if a cloud goes by, the voltage falls, and I'm no longer in absorb, and there is no way to tell if you are or aren't in absorb mode

A simple indicator or two would certainly clear this up. And, I understand your thinking about get away from the idiot lights in the first place. That does make sense.

I'm enjoying the snot of of CCDM, so much that I'm thinking of getting a large monitor and a cheap laptop for the power room in the garage. I'd let it run all the time.

mike

Maybe get a raspberry pi - it will run on less power .
I might try to set up the CCDM on my pi just for the fun of it.

Larry

I looked at them last night. It appears they are quite robust for such a small footprint. More 'computer' than an Ardunio. This will require much more reading on my part. However that being said, it would appear a $40 slice of pi and a monitor would be all I'd need for a dedicated Classic monitor.
Shooting from the hip, and I admit I haven't looked too deep, but I'm not sure how one gets the CCDM software into the raspberry pi? Load it on a SIM card? External HD of some sort? More questions than I have answers to.

Mike

If  you get a pi and get it set up and running then following the directions Brent gave should be pretty straight forward.
Or if you get to that stage and get stuck I could maybe remote in and help you with it.
The other way would be that someone ( maybe me ) could make a fresh version of it on an SD card and then make copy and all  you would have to do it flash it on an SD card ( and enter in your wifi ) .
Linux is a whole different world - took me awhile over the years to figure out how it works. But with the new desktop interface it is more like a regular computer.
Did you ever get an android and get Grahams Classic monitoring app going ? That really is a lot easier to do in many ways for monitoring until Brent gets his version all polished up. 
Larry

[qrper]

Brent,

Answers for some of you questions.

The display on my laptop is its default of 1366 x 768

I can't believe we have a day of bright sun, so CCDM is going wild with numbers. That's kinda cool looking at all the parameters change by the second. However, I can't tell if the Classic went to absorb or not unless I go the MISC screen and see if the absorb reached today is yes or no.

So, on the main page, there is no way to tell if the absorb voltage has been obtained, unless you know what the absorb voltage is. In my case, it's 58.8 V.
Now that being said, if a cloud goes by, the voltage falls, and I'm no longer in absorb, and there is no way to tell if you are or aren't in absorb mode

A simple indicator or two would certainly clear this up. And, I understand your thinking about get away from the idiot lights in the first place. That does make sense.

I'm enjoying the snot of of CCDM, so much that I'm thinking of getting a large monitor and a cheap laptop for the power room in the garage. I'd let it run all the time.

mike

Maybe get a raspberry pi - it will run on less power .
I might try to set up the CCDM on my pi just for the fun of it.

Larry

I looked at them last night. It appears they are quite robust for such a small footprint. More 'computer' than an Ardunio. This will require much more reading on my part. However that being said, it would appear a $40 slice of pi and a monitor would be all I'd need for a dedicated Classic monitor.
Shooting from the hip, and I admit I haven't looked too deep, but I'm not sure how one gets the CCDM software into the raspberry pi? Load it on a SIM card? External HD of some sort? More questions than I have answers to.

Mike

If  you get a pi and get it set up and running then following the directions Brent gave should be pretty straight forward.
Or if you get to that stage and get stuck I could maybe remote in and help you with it.
The other way would be that someone ( maybe me ) could make a fresh version of it on an SD card and then make copy and all  you would have to do it flash it on an SD card ( and enter in your wifi ) .
Linux is a whole different world - took me awhile over the years to figure out how it works. But with the new desktop interface it is more like a regular computer.
Did you ever get an android and get Grahams Classic monitoring app going ? That really is a lot easier to do in many ways for monitoring until Brent gets his version all polished up. 
Larry

Larry,

Yes I did!

I went to best buys and threw down the $$$ for a android table. Grahams interface is slick, but you know, I like Brent's too. Grahams looks great, and I sure love the meters, but the amount of information you get from Brent's software is simply amazing.

Now, as for the raspberry pi...

I'm still head scratching here. Lunix is totally new to me. Does the operating system have a web browser in it? I mean how does one get Brent's software downloaded into the raspberry pi?

Do I download the software as a zip file and stick it on a sd card? Then load it into the pi?

it sure is a learning curve that i"m willing to take.
mike

[ClassicCrazy]

You get the pi , then you download Raspberry Buster operating system from the website. You use a free program called Balena Etcher and that will flash the Buster OS from your download on computer to an SD card. Then you put the SD card in the pi and plug it in a  monitor, keyboard, and mouse   and you are in business - yes there is a regular window like desktop .
There are plenty of tutorials on youtube on how to do this. I like Explaining Computers youtube page but there are lots of others.
Once you get the pi up and running you don't even need the monitor or keyboard - there is a way to view everything on it from your Windows computer. As long as it is on your local network you can easily access it.
So to put in Brent's program on the Pi  you would just open a Cmd window pretty much like you did on Windows  and follow his directions.
This is how to put Ubuntu on raspberry pi 4 - you can use the same process to put on Raspbian Buster instead .
https://www.youtube.com/watch?v=RGaAZtA0ISQ
The only other consideration is that you can get a raspberry pi cheap but then you will need to get a power supply , case, SD card etc for it.
The old raspberry pi would run on any 5v supply but the newer ones need a bit more power so needs a decent 5v supply and one from a phone might not work
Rasberry Pi just came out with a new Pi 4 all mounted up in a keyboard and has the power supply ( I think) and operating system all installed. Might be a better option by the time you get all the other accessories for regular pi unless you already have them laying around anyway.
https://youtu.be/P1E5xszQqV8

Larry

[Fort Wisers]

Hi everyone! Happy New Year!

Mike-> glad to hear you're enjoying it, if you bump up your laptop's resolution you'll eliminate some of the graphics overlap.
A Pi really is just like a small Linux PC, and with the state of it's desktop it doesn't take long to get the hang of.
We have run CCDM on 3B+ and 4 series PI's with no issue.
They have USB ports so you can just straight copy the entire CCDM package onto the PI's desktop.
They typically come with python preloaded so generally all you have to do is load Pillow (similar to done in Windows) then copy CCDM to the desktop and edit the config file.

Mike and Larry -> Since you're both talking about trying it, one thing I have noticed bringing CCDM to a PI is there are some paths in the config file that might require the full paths to be specified (the default in the config file is relative paths) depending on what release of Rapsbian you run. Maybe it's a security thing, I never dug into it. I believe Stretch wanted full paths specified and Buster was fine as is (relative paths) but don't quote me on that. If you decide to try it and run into trouble getting it off the ground, let me know and I can walk you through the path edits, it will take less than a minute.


Mike, back to one of your previous comments; As for seeing Absorb on the Overview Tab there actually is a text field showing the current charge stage (below shows a system in Bulk, obviously):



So if the Classic goes into Absorb, you will see it here, if it drops back to Bulk (sudden cloud cover as you mentioned) you will see this charge stage drop back if the power being generated is not enough to maintain the Absorb voltage.
But you're correct, it doesn't stand out, so you've brought up a great point that we can work on to improve.
Let me look at the LED color scheme for some other controllers that use LEDs only for charge stage visualization and we can set the back ground color of this charge stage text field to match.
How does that sound?


Cheers, as always, keep the comments coming.
I'm hoping to set a target date for a number of the updates we've talked about soon......

Anyone else out there want to try what Larry and Mike have accomplished?

Cheers:
Brent

[ClassicCrazy]

Brent ,
I dug out my Pi3B+ but then got into other projects on it instead ! ( using sdr on a pi )
I already have Graham and Matt's Classic to MQTT program on it.
If you could make a docker install like Matt did - that would be great. Makes installing a lot easier for sure.
Andreas Spies and others have a docker install tool that will install those and other programs . It is called IOTstack
https://sensorsiot.github.io/IOTstack/Getting-Started/
So even though I dug out my pi to install CCDM on it - I haven't gotten to it yet .
I did find out a new tool that makes life a lot easier for me on the pi. I have Kitty for terminal program which is nice. But I also put on Notepad ++ which will connect to the pi ( with a plug in ) so you can look at files much easier than using terminal program. I also got another add on for Notepad ++ which will compare two different files. I had messed something up so was able to open the original I had on windows and the one on the pi - then click compare and it showed the two with any differences all colored.
I know you programmers probably have and use all kinds of tools like this - but this was new to me and made life easier .
I also have used the pi desktop via VNC Viewer but it seems kind of sluggish - I need to plug direct into the pi and see if it is different than going through vnc viewer.
So I will get back to the CCDM on the pi one of these late nights .

Larry

[Fort Wisers]

Morning Larry,

Let me see if I can free up some time to check out the link you sent and investigate docker, thanks for posting!

Yes, finding the time always seems to be an issue doesn't it?
Good luck with your PI....

Brent

[ClassicCrazy]

Morning Larry,

Let me see if I can free up some time to check out the link you sent and investigate docker, thanks for posting!

Yes, finding the time always seems to be an issue doesn't it?
Good luck with your PI....

Brent

Probably not worth the time right now figuring out how to build docker container - the time is probably better put into making all the refinements you want to the CCDM .
Docker is nice but  probably takes some time to build a container  - learning curve. I am only a happy end user of others efforts !
But the IOTstack tool makes it easy to install other programs on a pi - such as Mosquito , Node Red, Influx, Grafana , etc .
I am running CCDM today now since the sun is out.
I think I will see if I can get the CCDM installed on my pi now.
Larry

[ClassicCrazy]

Here are my notes for installing CCDM on Raspberry Pi in case it helps others.
First I want to point out that CCDM will not connect to the Classic if another program is using the Classic on IP such as the Midnite Local Status app or Android Classic Monitoring app. Only one IP connection at a time will work . ( it okay to have MyMidnite running and doesn't effect MyMidnite data getting sent out from Classic )

I am using a Raspberry Pi 3B+
running Buster Raspian 

I followed the instructions in Documentation in  CCDM package from Github
Using  checks python , pip , and pillow  I found everything already installed.
I am using SSH from my windows 10 computer and  terminal program called Kitty connected to the Pi ( same process  should work if you are running in a terminal on the pi desktop  )

To download files from github use this command
git clone https://github.com/Brent-Lekx-Toniolo/Charge-Controller-Data-Manager-CCDM.git


navigate to software folder
cd Charge-Controller-Data-Manager-CCDM/Software

to edit config file to change to my Classic IP address
sudo nano CCDM_config.py

after editing
control X  and say yes to save changes and yes to save in file

I then went to Raspberry Pi desktop   used file manager and opened up CCDM software folder.  ( I am still using my Window 10 computer and went to desktop with VNC Viewer)
I double clicked on CC_Data_Manager.py
and a program called Thonny opened up and at the top is a > Run button which when I clicked it the CCDM program opened up and ran on desktop just fine

Larry

[Fort Wisers]

Hey folks, sorry for the silence these last couple days....

Nice work Larry!
I have not run CCDM through thonny, I've always run it directly through python through bash / terminal.
I would be interested to see if it still generates the daily log files  each night running via thonny, if you happened to let it run overnight, let me know.

Take care:
Brent

[ClassicCrazy]

Hey folks, sorry for the silence these last couple days....

Nice work Larry!
I have not run CCDM through thonny, I've always run it directly through python through bash / terminal.
I would be interested to see if it still generates the daily log files  each night running via thonny, if you happened to let it run overnight, let me know.

Take care:
Brent

I haven't left it on overnight yet. Or checked the saved logs - guess I did have it on one night past midnight.
The thing is I have to shut it off sometimes because I can't see all the info I need sometimes since I have two controllers or if I need to use Local Status App to change an Aux setting.
What I  want to do is figure out how to use Influx DB and Graphana to make graphs of days data . It will just take me watching some tutorials to know how to do it.
I guess it might be easier to do the graphing using mqtt output from Graham and Matt's raspberry pi program.
Something else to learn !
Larry