CCDM - Charge Controller Data Manager

[ClassicCrazy]

Larry,
Before anything, can you confirm what OS you're running (Win10 etc)?

You look to be basically there as some of the messages you're receiving in that last screen shot are from CCDM, meaning it's actually trying to get off the ground.
Now the next hang-up is it can't find night_icon.png, this is a little icon used in CCDM to display when the controller is asleep (in solar mode).
My first thinking is I somehow forgot this icon in the GitHib package, so I just did a fresh download from Github, unzipped (To most closely imitate what you're seeing on your end I did this on D:\  as well, as I have a second drive similar to what is sounds like you have) and it ran no problem.......

So, in your CCDM package, right where CC_Data_Manager.py file is located, can you see a folder called "resources", in there should be another folder called "images", can you confirm what icon images are in there, night_icon.png should be in there, among others?

It seems I should add some notes into the getting started guide about pip3, I must have either got lucky (it was already installed on this PC?) or I simply don't recall (possible as my memory can be hit and miss lol) having any issues with it but will add some notes as per your findings Larry.

Let me try a fresh start on my wife's pc this morning.............

Sorry gang, it certainly shouldn't be this difficult.
Thanks for the feedback so far, and for being patient as we get this hammered out.

That is okay - I contribute by testing these things and happy to help document or figure out the bugs .
Yes I am using Windows 10 on a new laptop I got recently .
Yes the Resources folder is there with images in it.
I tried moving the CCDM folder to C: drive but still same error .

Larry

[Fort Wisers]

Hey folks,
OK, I did a from the ground up install and am building a new Getting Started Guide.
I hope to have it complete in the next couple of days, hopefully it will help with everything.
I'll post back when it's ready.......
Merry Christmas everyone!

[ClassicCrazy]

Hey folks,
OK, I did a from the ground up install and am building a new Getting Started Guide.
I hope to have it complete in the next couple of days, hopefully it will help with everything.
I'll post back when it's ready.......
Merry Christmas everyone!

thanks for your efforts .
I spent some time trying to figure out why it wouldn't find the file in the folder.
Digging into some of the code and looked up errors helps me understand how these programs work. At least on the surface.
Enjoy your Christmas - sun is out here for a change so it is happy holidays !

Larry

[Fort Wisers]

Ok, hopefully you folks haven't given up on this project yet!
I have updated the repository and included a much improved (hopefully) Getting Started Guide, it can still be found in the Documentation folder.
So blow away any CCDM package you downloaded previous and start again with a fresh download from the repo (here):

https://github.com/Brent-Lekx-Toniolo/Charge-Controller-Data-Manager-CCDM

The big thing I noticed during my testing is allowing Windows Store to install Python caused the most grief as it doesn't let you customize the install.
So you'll see a specific note about this in the Getting started guide.

To summarize the Getting Started Guide, the general steps in Win10 are:

1. Check if python 3.4.x + is installed by typing "python3 --version" (with-out quotes) in the command prompt -> if not follow the link in the getting started guide and download the installer, then install as per the guide (avoid allowing Windows Store to install)
2. Check if pip3 is installed by typing "pip3 --version" (with-out quotes) in the command prompt -> if not then grab that installer as per above
3. Check that the Pillow package is installed by typing "pip3 list" (with-out quotes) into the command prompt -> if it's not in the list install it by typing "pip3 install Pillow" (with-out quotes) into the command line

Larry -> I know you're hoping to place CCDM onto your D:\ drive....this shouldn't be an issue. But to start, follow the Getting Started Guide and place the CCDM package onto your Desktop. If you get CCDM up and running, then move the entire thing to anywhere on D:\.

Larry and Mike-> Thanks again for spending time working on this and sending feedback of where the issues are. Hopefully I was able to capture enough in the new Getting Started guide that we're met with more success! I really appreciate all the feedback on this one.

Last note: You may end up having to uninstall previous installs of python the Windows Store installed for you......I'm not sure.
Finger's crossed!

Merry Christmas and Happy Holidays
Brent

[ClassicCrazy]

Ok, hopefully you folks haven't given up on this project yet!
I have updated the repository and included a much improved (hopefully) Getting Started Guide, it can still be found in the Documentation folder.
So blow away any CCDM package you downloaded previous and start again with a fresh download from the repo (here):

https://github.com/Brent-Lekx-Toniolo/Charge-Controller-Data-Manager-CCDM

The big thing I noticed during my testing is allowing Windows Store to install Python caused the most grief as it doesn't let you customize the install.
So you'll see a specific note about this in the Getting started guide.

To summarize the Getting Started Guide, the general steps in Win10 are:

1. Check if python 3.4.x + is installed by typing "python3 --version" (with-out quotes) in the command prompt -> if not follow the link in the getting started guide and download the installer, then install as per the guide (avoid allowing Windows Store to install)
2. Check if pip3 is installed by typing "pip3 --version" (with-out quotes) in the command prompt -> if not then grab that installer as per above
3. Check that the Pillow package is installed by typing "pip3 list" (with-out quotes) into the command prompt -> if it's not in the list install it by typing "pip3 install Pillow" (with-out quotes) into the command line

Larry -> I know you're hoping to place CCDM onto your D:\ drive....this shouldn't be an issue. But to start, follow the Getting Started Guide and place the CCDM package onto your Desktop. If you get CCDM up and running, then move the entire thing to anywhere on D:\.

Larry and Mike-> Thanks again for spending time working on this and sending feedback of where the issues are. Hopefully I was able to capture enough in the new Getting Started guide that we're met with more success! I really appreciate all the feedback on this one.

Last note: You may end up having to uninstall previous installs of python the Windows Store installed for you......I'm not sure.
Finger's crossed!

Merry Christmas and Happy Holidays
Brent

Thanks Brent - I figured the multiple Pythons I installed probably messed things up somehow.
I will  uninstall everything ( if I can ) and then start out from scratch on your new directions and package. Maybe later tonight before Santa gets here.

Larry

[ClassicCrazy]

Ok, hopefully you folks haven't given up on this project yet!
I have updated the repository and included a much improved (hopefully) Getting Started Guide, it can still be found in the Documentation folder.
So blow away any CCDM package you downloaded previous and start again with a fresh download from the repo (here):

https://github.com/Brent-Lekx-Toniolo/Charge-Controller-Data-Manager-CCDM

The big thing I noticed during my testing is allowing Windows Store to install Python caused the most grief as it doesn't let you customize the install.
So you'll see a specific note about this in the Getting started guide.

To summarize the Getting Started Guide, the general steps in Win10 are:

1. Check if python 3.4.x + is installed by typing "python3 --version" (with-out quotes) in the command prompt -> if not follow the link in the getting started guide and download the installer, then install as per the guide (avoid allowing Windows Store to install)
2. Check if pip3 is installed by typing "pip3 --version" (with-out quotes) in the command prompt -> if not then grab that installer as per above
3. Check that the Pillow package is installed by typing "pip3 list" (with-out quotes) into the command prompt -> if it's not in the list install it by typing "pip3 install Pillow" (with-out quotes) into the command line

Larry -> I know you're hoping to place CCDM onto your D:\ drive....this shouldn't be an issue. But to start, follow the Getting Started Guide and place the CCDM package onto your Desktop. If you get CCDM up and running, then move the entire thing to anywhere on D:\.

Larry and Mike-> Thanks again for spending time working on this and sending feedback of where the issues are. Hopefully I was able to capture enough in the new Getting Started guide that we're met with more success! I really appreciate all the feedback on this one.

Last note: You may end up having to uninstall previous installs of python the Windows Store installed for you......I'm not sure.
Finger's crossed!

Merry Christmas and Happy Holidays
Brent

I uninstalled or deleted all past copies of Python.
I followed the install directions exactly as in your new document.
But if I go to CMD or powershell
and do Python --version I get error. ( first screenshot)
If I find Python in the start menu with all the other programs and click it then it opens a CMD window and shows up . ( second screenshot)
I moved on to pip3 check and that was installed.
I checked for Pillow and that was not listed so I did the pip3 install pillow and that installed it.
checked and pillow was listed.
I went to Config and entered the IP of my Classic and saved it.
Then I double clicked on CCDM data manager and
It Worked !

So I guess the only note I would put in your instructions is that the check for Python in CMD might not work - it only worked for me by clicking on the Python link in the start menu.
Final  photo below is wish list for data in future ( taken from Grahams Classic Monitoring app )

Thanks again Brent for your efforts - good to have success at 1:30 AM - now I gotta hit the sack so Santa can come ( maybe - he finds some excuse to skip me every  year !)

Larry

[Fort Wisers]

I uninstalled or deleted all past copies of Python.
I followed the install directions exactly as in your new document.
But if I go to CMD or powershell
and do Python --version I get error. ( first screenshot)
If I find Python in the start menu with all the other programs and click it then it opens a CMD window and shows up . ( second screenshot)
I moved on to pip3 check and that was installed.
I checked for Pillow and that was not listed so I did the pip3 install pillow and that installed it.
checked and pillow was listed.
I went to Config and entered the IP of my Classic and saved it.
Then I double clicked on CCDM data manager and
It Worked !

So I guess the only note I would put in your instructions is that the check for Python in CMD might not work - it only worked for me by clicking on the Python link in the start menu.
Final  photo below is wish list for data in future ( taken from Grahams Classic Monitoring app )

Thanks again Brent for your efforts - good to have success at 1:30 AM - now I gotta hit the sack so Santa can come ( maybe - he finds some excuse to skip me every  year !)

Larry

Morning Larry!
WOW, thank you for spending such a late night working on this, especially on Christmas eve!
My thinking is that Santa will have rewarded you for your extra efforts!


OK onto business before the kids get up to see what Santa brought:
I'm very glad you got CCDM up and running, this is great news, and I am so thank full for your determination and feedback through this first step.
Let it run over a 24 hour period to get a feel for what it ends up logging, a new log file will be saved under the CCDM_logs folder each evening at 11:59pm

Onto your comments.....
Python Version Check After Install:
On one of my python install trials I encountered a similar issue to you for entering "python3 --version" after the install (as confirmation).
Except in my case the system simply returned nothing, no error, no version return, nothing......just kicked right back to the command prompt.
I left it as I knew the install was ok and found that later after a reboot the version check worked just fine.
So that's your first thing to check, trial the version check again after the device is rebooted, it's possible Window's PATH associations aren't updated until after a reboot? I don't know.
If that doesn't work then the second check is to run the python installer again and be sure the "Add Python 3.9 to PATH" check box is checked, as seen here:



Lastly if all of the above never produces the results we're looking for, try "python --version" instead (omitting the 3 at the end of python).
If it never get's solved yet you're running python applications just fine then absolutely we'll add a note into the getting started guide!

CCDM Data Points:
There is no doubt that Graham's Application is a nicely polished product.
Some of the data point's in your request screen shot are already there in CCDM.
For Example, if we navigate over to the Misc tab:



In the above you will see the following:
1. flag for indication of In V is lower than Out. -> I layed out and wrote the text for each flag identical to how they show up in the Midnite manual.
2. Time In Float Today -> Pretty self explanatory, if the system hits float then the duration in float is shown here. This is a value taken from modbus reg 4138

3. Time In Absorb Today -> this one takes a bit of explaining. In the Classic controllers there is a data point at modbus reg 4139 and it is an up / down counter.
The way I understand it, each day this counter starts at your absorb time setpoint (so in your case it shows as 4:00:00 (IE 4 hours). Then when the system hits absorb setpoint voltage the timer starts counting down.....if it hits zero OR the end amps threshold is reached (IE current going into the battery drops below end amps), then the system switches to float as the battery bank is considered charged. For me personally, I thought, the count down value means little to me, what I really want to know is how long did I spend in absorb (similar to how float is displayed). So I do a little math and subtract this absorb timer value from the Absorb Time Setpoint and that gives us the Time in Absorb Today.
I hope that makes sense, lot's of absorbing to do there (haha bad humor for Christmas morning).

Now head on over to the Configuration Tab:



Here you can see the following:

1. Absorb Time -> this is the absorb time setpoint, perhaps I'll add "setpoint" to this text so it's a little more clear as to what this is
2. Absorb End Amps -> Ending amps used in conjunction with absorb time setpoint to decide when it's time to switch to float
3. EQ Time -> this is the EQ Time setpoint, again the addition of "setpoint" in the text will help clear this up.

Now what I am missing compared to Graham's application is a note on reason for resting, a note on battery target voltage and just a general more text based method of showing all this.
Let me first add in the reason for resting and target voltage data points.
A more polished look and feel will hopefully come with time as we work on developing this as a whole.
I will open the above notes (add setpoint text and two data points mentioned) as issues on our GitHub repo....


A user guide might be worth putting together as well (already on the issues list in GitHub) to explain the data points that are there. Mind you the more intuitive we can make the whole system the better.......
I added an appendix at the bottom of the Getting Started guide that has a quick little blurb on each tab and it's contents but it doesn't go into great detail of what each data point is.....so a more in-depth "User Manual" might be in order for this....

OK, I'm now just rambling and typing out my thoughts so it's best I go.
Thanks again Larry for sticking it out, hopefully other's will get adventurous and give it a trial.

To everyone, go pour and extra coffee (perhaps with some Bailey's) and enjoy, Happy Holidays to all:
Brent

[ClassicCrazy]

Brent
Python --version  worked and showed me Python version okay
Python3 --version gives me the error

I have to study all the data on CCDM - there is a lot going on there !
One thing I have not found so far is the equivalent of Net AH which is on both Local Status app under Whizbang
and in Grahams android app on the Capacity screen ( the dial there is labeled Net Charge )
It shows when everything that has been taken out of the battery is put back in.

I was just looking around in the code - don't know anything about Python but interesting to see how it all is laid out.
That was a lot of work writing all that !
I was curious if I could spot where each modbus register is sorted out - but maybe that is in the compiled sections ?
My mission was to see if I could change a Celsius temp to Fahrenheit  just for the fun of it ( and since that is how my brain interprets temps). I didn't spot anywhere I could put the formula in - but I did see how to change the labels . Not a big deal is just a fun exercise to see what makes this stuff tick. 
   
Santa left me a super sunny day here so I can make some hot water with lots of extra power.

Larry

[Fort Wisers]

Hey Larry!

You are correct, I do not have net amp hours on the UI, consider it in the works.
I have always just relied on SoC but you're correct, net amp hours is a more accurate way of understanding what has gone in versus what has come out of the bank, so an important point to display.

All the data comes in from the modbus reads in the CC_Class.py code.
The compiled files (under _pycache_ folder) are just files python creates during the runtime (IE when the application is launched), 100% of the code is open source.
Adding a conversion to deg F is a great idea!

I shall add both ideas to the issues list.
If you ever want to see what is currently "in the works" you can see that here:
https://github.com/Brent-Lekx-Toniolo/Charge-Controller-Data-Manager-CCDM/issues
Issue#10 lists the data points to be added as per your feedback, issues#11 is for the ability to change between degrees C and F.
You can click on each issue to get more detail.
My plan is to add these in with 0.0.2 release, you will see a number issues (classified as enhancements) listed with a v0.0.2 milestone, no target date yet though.

Cheers, thanks again for the feedback, enjoy the sun and hot water!

Brent
P.S. since you're having a sunny day, send a screen shot of your peaks and events (misc tab) if you get a chance

[qrper]

It's not when you do it all the timeâ€"and I couldn't locate a 13 year old to help me.

I haven't poke around with computers since my Apple II days.  At one time, I could keep my head above water with DOS, but crap, that's been 25 years ago. Heck at one time, I use to build my own pc (Who didn't?) now, when a HD dies, I take it to the local computer guy and have him install a new SSD and move all the files. I mean he's got kids to feed, too.


Now that you told me where to enter the commands, (I had to look up how to get to the command line on windows) I'll give it another shot today. I'll also take a peek at the resource folder and see if that file is there.

Hey Mike,

We're all doing our best to keep on-top of technology.
If I had made a better, more complete getting started guide perhaps a lot of the issues you guys are struggling with could be avoided.
So I appreciate you guys starting with me at ground zero as we get together something a little more user friendly......

Hopefully it ends up being a package that many find useful, and if not, then we try again!
I appreciate all the feedback from you and Larry, best of luck when you go to try again.
In the meantime I'm going to work on that guide and get it further along, I see python has a few more options in their 3.9 installer and I really should add a few notes on pip3 as well.
Spare time to work on personal interest projects / hobbies always seems to be the issue!

Take care everyone:
Brent

Poked around a bit more. Got the window command panel up. I forgot all about that guy.
However, when I type in ;

pip3 install pillow

I get a error that says pip3 is not recognized as internal or external command

I do have python that appears to be running, I can open it and see it's screen.

however!! If I type python within the CMD screen, I get an error that 'Python is not recognized as a internal or external command."

It's the holidays... so don't put any work into this. I'll continue to poke at it and see what's going on

mike

Mike ,
How did you install the Python ?  When I used the Microsoft app store to do it everything seemed to start working .
Larry

Larry,

I went to the site posted and downloaded python. I clicked on the install button and let it run
I click on the python icon and it opens a window (looks like the cmd window)

mike

[qrper]

Holy snapping electrons batman!

I got the software up and running!

Brent, this version of the install docs are a zillion times better.

A couple of notes.

I did a clean install on my laptop running windows 10

No matter what I did, windows wanted me to let it install python.

After I did the python install outside of windows, when I typed in python at the cmd prompt, it wanted to install python. So I keep poking around and at last  I had it installed.

Everything went more or less as planned, except my IP was one of those 10.1.1.0 things, edited the config file and whoa! There be CCDM on the laptop.

CCDM was running, but nothing much was on the screen. I had a connection, but the python dialog box was filled with errors. So....what to do?

Restart the damn computer!

And bingo! There be smiles here.

No all I need is some sunlight!

Damn nice Christmas present, too

mike

[ClassicCrazy]

Holy snapping electrons batman!

I got the software up and running!

Brent, this version of the install docs are a zillion times better.

A couple of notes.

I did a clean install on my laptop running windows 10

No matter what I did, windows wanted me to let it install python.

After I did the python install outside of windows, when I typed in python at the cmd prompt, it wanted to install python. So I keep poking around and at last  I had it installed.

Everything went more or less as planned, except my IP was one of those 10.1.1.0 things, edited the config file and whoa! There be CCDM on the laptop.

CCDM was running, but nothing much was on the screen. I had a connection, but the python dialog box was filled with errors. So....what to do?

Restart the damn computer!

And bingo! There be smiles here.

No all I need is some sunlight!

Damn nice Christmas present, too

mike

Mike ,
I was getting the same thing - when I typed in Python on CMD it would open up internet explorer and take me to the Microsoft app store.
I found these settings which is what I think disabled that when I turned them off.
Not sure if they are on every Windows 10 now ?
Maybe Brent can add this to his instructions to shut it off to avoid confusion .
Anyway glad you got it working - that is a great feeling !
see attached photo for the Setting to turn off the Python auto Microsoft store thing ( though you probably won't have the issue any longer now that you have it installed).

Larry

[ClassicCrazy]

Brent
Here is screenshot of the peak.
I didn't have the program on all day so that is why it doesn't show true Absorb time.
Sun went away and then came back in the nick of time of finish heating water.
I turned on a bunch of loads to max out controller so I could capture the peak in attached screenshot. This is from 2700 watts solar array .
I have two controllers in Follow Me . They both share the same Whizbang shunt . Not sure how your program deals with that right now.
But it does seem like the peak reported below was just from the one controller.

Larry

[Fort Wisers]

Morning folks!

Mike -> great job getting your system up and running! I'm glad the new guide seems to be better.
I will add a note about rebooting ad I had to do the same during one install trial.

Larry -> I can indeed add notes about the settings you found and am going to add a note about the python --version (vs python3) command as per your findings as well.
In terms of Follow Me scenarios, CCDM would only be looking at what ever is reported by controller 1, I have not finished up the code for multiple controllers.
Now, I'm not 100% familiar with how follow me works internally, if controller one reports the summation of all following controllers? I don't know but will read up on it.
I guess in the case of a shared whizbang you should be seeing the current of both controllers.......anyhow I'll have to give that scenario some thought.
And I'll have to monitor peak whizbang current as well since this would be additive of the two controllers in your case.
My first step will be to get multiple controller code finished and then handle follow me scenarios and do an auto detect of a set of controllers in follow me mode.....



I'm not sure if I'll get to any work on it done today, then tomorrow we head to our cabin for a week and will be off line for the most part.
We have internet there but I tend to not look at the phone much while in the bush and have no pc up there.
So I'll check back in early in the new year and get some of these updates completed.

Guys, thanks again for the feedback and sticking with it......
Take care and Happy New Year to all:

Brent

[qrper]

Morning folks!

Mike -> great job getting your system up and running! I'm glad the new guide seems to be better.
I will add a note about rebooting ad I had to do the same during one install trial.

Larry -> I can indeed add notes about the settings you found and am going to add a note about the python --version (vs python3) command as per your findings as well.
In terms of Follow Me scenarios, CCDM would only be looking at what ever is reported by controller 1, I have not finished up the code for multiple controllers.
Now, I'm not 100% familiar with how follow me works internally, if controller one reports the summation of all following controllers? I don't know but will read up on it.
I guess in the case of a shared whizbang you should be seeing the current of both controllers.......anyhow I'll have to give that scenario some thought.
And I'll have to monitor peak whizbang current as well since this would be additive of the two controllers in your case.
My first step will be to get multiple controller code finished and then handle follow me scenarios and do an auto detect of a set of controllers in follow me mode.....



I'm not sure if I'll get to any work on it done today, then tomorrow we head to our cabin for a week and will be off line for the most part.
We have internet there but I tend to not look at the phone much while in the bush and have no pc up there.
So I'll check back in early in the new year and get some of these updates completed.

Guys, thanks again for the feedback and sticking with it......
Take care and Happy New Year to all:

Brent

Thanks, Brent
You gotta understand this is from a guy that took two hours to get the LED to blink on an Ardunio.

While I haven't had a lot of time to play with CCDM, I will pass on a few observations.

1. The graphics (on my laptop) are covered by the data panels (widgets?) The battery icon is barely visible.
2. On the python window, I see a lot of errors flying by. They go so fast, I can't read them. It says something like:error reading reg4106-4108 block of charge controller.

There are maybe ten lines like this with different errors on different blocks.

While Mr Sun hasn't shown his face for over two weeks, having peaking reading values is well worth the price of admission.

On my wish list..

there is so much text, How about a panel, all it system status or something like that with some simple idiot lights for us idiots. Like 'charging' absorb, float and so on.

I guess it's the geek in me that can sit and watch numbers tumble around on a computer screen all day.

You have an amazing piece of software... Enjoy the down time at the cabin.

mike