Just to give an idea where this is heading heres a screenshot of the default view screen. It updates every minute. By clicking setup you can specify which datapoints you want to see where, and which series to plot on graphs. You can add your own templates to represent the data pretty much anyway you want.
It takes the cubieboard 2 seconds to render that page. And thats with most of the heavy lifting having been done previously by a cronjob. The plan is to remove the database, because it isnt adding anything except overhead.
The code and a demo will be up in the next few days. At that point its wireframe / proof of concept only and anyone that wants to help shape its evolution will be more than welcome to chip in.
PS. see the flat spot betw 7-8am. Thats when the classic's ethernet locked up, only fixable by rebooting the classic.
That looks fantastic! Excellent work. I'm sorry about the Classic Ethernet issues.
Zoneblue what modbus library are you using? And not sure how well its documented but the older code 1401 and back had a 30 second inactivity timer on the tcp/ip stack after no modbus reads for 30 seconds it will close the connection. The new Beta code on the website has a 5 minute inactivity timer.
Ryan
Project site is now up: http://code.google.com/p/theblackboxproject/
For just now youll need svn. Remember the code is stilll way, way, pre release.
re. firmware, Im still using 1370. I will try 1549 if you think it will help.
Quote from: Halfcrazy on September 16, 2013, 07:26:08 AM
Zoneblue what modbus library are you using?
I have tried them phpmodbus, pymodbus and newmodbus. The latter is by far the least affected by this issue. The reason is that it is the lightest and quickest and the issue to my mind is a timing thing.
I dont know if you recall but back with 1070 and the local app at the time it had these exact same symtpoms, random disconnects/fail to connect /timeouts. Something about timing has been tweaked because with 1370 and the local app of its era, the locall app disconnects went away.
If you want to reproduce this issue, its most likely that installing the black box on your lab rPi will do it pretty quick.
Quote
And not sure how well its documented but the older code 1401 and back had a 30 second inactivity timer on the tcp/ip stack after no modbus reads for 30 seconds it will close the connection. The new Beta code on the website has a 5 minute inactivity timer.
Ryan
Wasnt that for outgoing http posts, for dns timeouts etc? Im stuggling to see how that is related to incoming modbus connects? But i will try it if it helps progress this.
The 30 second disconnect is a safety thing. TCP/IP has no built-in keep-alive mechanism (although some stacks implement it as a hack). That is left up to the application stack. MODBUS also does not specify a keep-alive mechanism. The Classic, since it only has one precious listening port, checks for activity on the port. If it is silent for more than 30 seconds then it automatically terminates the connection. This means you must ask for at least one register (valid or not) at least once every 30 seconds or your application will be disconnected.
Interesting. An easy wasy for me to test that is to shorten my sample interval from 60s to 20s.
Id assumed i was making brnad new connections every 60 seconds. So, when 60s later you make a new request, what happens? Im getting a mixture of connect refused, and timeouts. Whats making those odd connection fails, and then later the ethernet port shut scompletely?
Out of interest , how frequently can you make modbus over ethernet requests? Without impacting on the controllers resources.
Quote from: zoneblue on September 17, 2013, 06:24:34 PM
Id assumed i was making brnad new connections every 60 seconds. So, when 60s later you make a new request, what happens? Im getting a mixture of connect refused, and timeouts. Whats making those odd connection fails, and then later the ethernet port shut scompletely?
For what it's worth, my newmodbus when run in loop (continuous) mode will leave the connection open and just keep making queries at the appropriate time. You can force it to close and re-open the connection with the -r switch (Re-open) which instructs it to open, read/write, close. This was mainly intended to help it co-exist with a local-app on the same network, although I don't know if the local-app closes the connection or just hogs it :)
The Local App is a resource hog! It grabs the connection and tries never to let go. If a communications error causes it to shut down the connection then it makes a grab for it again as soon as possible. Greedy, greedy, greedy. RossW's connect-poll-disconnect architecture is a much more altruistic method and definitely the one to emulate. I would attempt to do the same on the Local app but it would mean tearing it apart and leaving it scattered all over the yard for a few months. By then I'd be too lazy to put it all back together and I'd by a used Miata kit instead.
As far as the Classic goes, you can poll for registers as fast as it can give them to you. I've run modscan down to 50ms. If you are opening a connection and leaving it open and then opening another you might be running into problems on the PC side rather than the Classic. The Classic _should_ just ignore the new connection request although different IP stacks sometimes employ sneaky ways to take over a connection some of which the Classic's tiny stack can't currently cope with.
The recommended way to talk to the Classic is to open a connection to it, run your register poll, then close the connection.
You can also do what the Local App does which is to open a connection and then poll for registers at least once per every 30 seconds down to once every 50ms if you need that resolution. You might start getting modbus errors at that speed though depending on how loaded the communications stack is.
If you need to write registers then the second method is a little easier since you will have to write the unlock registers every time you open a new connection to the Classic. You can disable this behavior by installing the "Lock" jumper, though.
Keep in mind that you should be doing full error checking when doing TCP/IP communications. There are a lot of things that can go wrong and if you aren't keeping track of IOExceptions/Network Errors/TCP Errors/etc, then you may be doing things like attempting to write data to a closed or partially open connection. This may have bad consequences on the Classic if your modbus engine is trying to alert you that there is an error on the bus but you are still trying to write data to a partially open connection.
For example if you open a connection to the Classic and then wait 60 seconds the Classic will timeout after 30 seconds and attempt to close the connection. If you ignore the close indication in your application and simply try to write to the existing connection then the behavior is undefined. Your PC stack _should_ close the connection as per the Classic's request but some leave it partially open as an optimization in case you want to re-open it later. Case in point: the Local Application keeps its connection open partially because you cannot fully close a TCP socket under AIR SDK 2.5 without shutting down the application. What this means is that as far as the Classic is concerned a connection may be closed, but the PC still keeps it open in order to speed up the process of re-opening it. It does alert the application code by throwing an IOError event which indicates that the connection has been closed by the other end which I then use to mark the connection "Closed" in my application code and then stop writing to the socket. If I ignore that "IOError" event then my application can still write to the socket, the stack assumes I know what I'm doing and then puts the data on the bus. If the Classic has not finished its internal socket cleanup code then some of the data may get processed which leaves the socket in a bad state. I have attempted to address this issue in the firmware, but there are still some small border cases in which the connection can still get confused. If your code is written in Java or uses .NET or Python (or AIR) then this is a possibility.
It is best for your application to honor and process the error signals coming from your modbus stack. If the modbus stack is not sending Close or error events then you should probably use a different stack.
Andy,
Thanks for your consdered response on this. I see that for connections that are kept open, there are some factors to consider there for sure. Are you saying, that in general we /should/ be aiming to keep a connection open?
The model i am using is the same cautious approach that newmodbus takes by default. Open, get the data, close. Repeat 60s later. Same reason, to allow local app to work, for configuration purposes. Is there something about this methodology that is problematic for the classic? Does it take a long time to reissue new connections?
Newmodbus is very very fast, an order of 10 times faster than any other method. The fact that the data doenst hit the db until 2 or 3 seconds past the minute is caused solely by my post processing.
Its not ideal that when the local app is running i lose data, but in the scheme of things its something we can live with.
But something is going on somewhere. The current blackbox trunk code results, on the A10 cubiebaord, in 1 failed connect per hour roughly, and a complete lockout about once every 30 hours. You can tell by the timestamp in the db whether it timed out, or was refused. Lately they are more the refused tpye. Refused entrys appear as 2 seconds past the minute, timed out entrys appear 32 seconds past the minite, as newmodbus appears to retry for 30 seconds then giveup.
As i have indicated is very much depends on the load on the board as to how bad the problem is. I have i believe ruled out a bug in newmodbus, something like a connection left open, as my test using /modules/midnite_classic/readclassic.py exhibits refused connections at about 4 times worse a rate.
Heres the php from /modules/midnite_classic.php: line 501 ($binary is newmodbus, and that function is called from cron each minute, so its a new process each time):
//invoke the binary and parse the results
if ($this->debug) print "\nInvoke binary: $dir/$binary $ip";
exec("$dir/$binary $ip 16385-16390 4101-4340", $lines,$ret);
if ($ret) {
$this->error= true;
if ($this->debug) print "\nRead device: FAIL";
return false;
}
else {
foreach ($lines as $line) {
if (!preg_match("/^\d\d\d\d\d? /",$line)) continue;
list($register,$value)= explode(" ",$line);
$this->registers[$register]= $value;
}
if ($this->debug) print "\nRead registers: ".count($this->registers);
}
Just to repeat for clarity. On an intel atom dual core mini itx board, this problem does not occur at all. But I have previously seen issues with local app disconnects on netbook computers which i believe is directly related. On the cubieboard, the failed connects occur when the board is loaded. If there are no web pages being served by lighttp for instance the script talking to classic gets left alone to run more or less happily. The cronjob is the only script that talks to the classic, the webpages deliver data that had been previously stored in the database. That combined with the 2s per 60s cron constraint, means there are no chances of overlapping connect attempts.
I have not tried using newmodbus's regular write to disk function.
Edit: i just had a thought. The once per hour connect refused may be related to a rolling overlap between newmodbus running and a web browser page render. Ie the view page refreshes every minute, but becasue of page delivering overhead, its going to take say 61 seconds, therefore periodically overlap the cron 60s, about once an hour... That means, other than the extra load on the cpu, extra *network traffic*. Could this be a clue? Ill leave the view screen off today, to confirm.
Bang goes that theory, it seems particularly bad this morning:
id state tbat tcc pout vpv ipv vout iout date_created infoflags code
----------------------------------------------------------------------------------------------------------
36140 4 10.1 34.0 478.0 76.2 6.1 27.2 17.4 2013-09-19 09:38:04 838873092 0
36139 4 10.1 33.8 229.0 83.6 2.5 27.1 8.4 2013-09-19 09:37:04 838873092 0
36138 4 10.1 33.6 523.0 76.6 6.7 27.2 19.1 2013-09-19 09:36:04 838873092 0
36137 4 10.0 33.3 276.0 82.1 3.2 26.9 10.2 2013-09-19 09:35:04 838873092 0
36136 4 10.0 33.0 342.0 76.9 4.3 26.9 12.7 2013-09-19 09:34:04 838873092 0
36135 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:33:04 0 1
36134 4 10.0 32.9 326.0 76.2 4.1 26.9 12.1 2013-09-19 09:32:03 838873092 0
36133 4 10.0 32.9 450.0 74.3 5.9 27.1 16.5 2013-09-19 09:31:04 838873092 0
36132 4 10.0 32.7 425.0 76.3 5.5 27.0 15.7 2013-09-19 09:30:04 838873092 0
36131 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:29:04 0 1
36130 4 10.0 32.2 297.0 74.6 3.8 26.6 11.1 2013-09-19 09:28:03 838873092 0
36129 4 10.0 32.3 235.0 75.7 2.9 26.6 8.8 2013-09-19 09:27:04 838873092 0
36128 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:26:04 0 1
36127 4 10.0 32.2 286.0 72.9 3.7 26.7 10.7 2013-09-19 09:25:03 838873092 0
36126 4 10.0 32.1 266.0 74.1 3.4 26.7 9.9 2013-09-19 09:24:04 838873092 0
36125 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:23:04 0 1
36124 4 10.0 32.1 427.0 74.9 5.5 27.0 15.7 2013-09-19 09:22:03 838873092 0
36123 4 10.0 32.0 406.0 75.5 5.2 26.9 15.0 2013-09-19 09:21:04 838873092 0
36122 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:20:04 0 1
36121 4 10.0 31.8 376.0 75.8 4.8 26.9 13.9 2013-09-19 09:19:03 838873092 0
36120 4 10.0 31.7 383.0 77.3 4.8 26.9 14.2 2013-09-19 09:18:04 838873092 0
36119 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:17:04 0 1
36118 4 10.0 31.2 329.0 69.3 0.0 26.8 0.0 2013-09-19 09:16:03 838873092 0
36117 4 10.0 31.2 356.0 74.2 4.6 26.8 13.2 2013-09-19 09:15:04 838873092 0
36116 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:14:04 0 1
36115 4 10.0 30.9 219.0 76.5 0.0 26.8 8.2 2013-09-19 09:13:03 838873092 0
36114 4 9.9 30.7 222.0 78.1 0.0 26.8 8.3 2013-09-19 09:12:03 838873092 0
36113 4 10.0 30.6 373.0 76.1 4.8 26.8 13.9 2013-09-19 09:11:04 838873092 0
36112 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 09:10:04 0 1
36111 4 9.9 30.1 127.0 82.8 0.0 26.5 0.0 2013-09-19 09:09:03 838873092 0
36110 4 10.0 30.0 327.0 73.7 4.2 26.5 12.3 2013-09-19 09:08:03 838873092 0
36109 4 9.9 29.8 94.0 81.6 0.0 26.3 3.6 2013-09-19 09:07:03 838873092 0
36108 4 9.9 29.6 89.0 83.0 1.1 26.3 3.4 2013-09-19 09:06:03 838873092 0
36107 4 9.9 29.4 81.0 82.1 0.9 26.1 3.1 2013-09-19 09:05:03 838873092 0
36106 4 9.9 29.3 78.0 83.4 0.8 26.0 3.0 2013-09-19 09:04:03 838873092 0
36105 4 9.9 29.0 65.0 82.9 0.6 25.9 2.4 2013-09-19 09:03:03 838873092 0
36104 4 9.8 28.9 62.0 81.6 0.5 25.8 2.4 2013-09-19 09:02:03 838873092 0
36103 4 9.9 28.8 51.0 80.9 0.4 25.7 2.0 2013-09-19 09:01:03 838873092 0
36102 4 9.9 28.8 72.0 80.6 0.6 25.8 2.8 2013-09-19 09:00:03 838873092 0
36101 4 9.8 28.8 145.0 72.5 1.8 25.8 5.6 2013-09-19 08:59:03 838873092 0
36100 4 9.9 28.8 150.0 72.5 1.9 25.7 5.8 2013-09-19 08:58:04 838873092 0
36099 4 9.8 28.8 145.0 73.0 1.7 25.7 5.6 2013-09-19 08:57:04 838873092 0
36098 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2013-09-19 08:56:04 0 1
36097 4 9.9 28.7 108.0 78.1 1.2 25.7 4.2 2013-09-19 08:55:03 838873092 0
36096 4 9.9 28.6 165.0 74.9 2.0 25.6 6.3 2013-09-19 08:54:04 838873092 0
36095 4 9.9 28.5 138.0 74.3 1.6 25.6 5.4 2013-09-19 08:53:04 838873092 0
I'm not sure what is happening then. I wonder if RossW or Tom could chime in if they've seen similar issues?
Quote from: atop8918 on September 19, 2013, 03:53:33 AM
I'm not sure what is happening then. I wonder if RossW or Tom could chime in if they've seen similar issues?
I've been running newmodbus since shortly after I got my classic late last year. It polls my classic every 300 seconds from one of my FreeBSD boxes (a non-descript, low-power 1.2GHz box) but I have never, not once, experienced this problem without direct and deliberate provocation (like intentionally running multiple instances concurrently).
The only problem I've experienced has been the 2 to 9 times a day spontaneous reset, which "went away" after my classic blew up and we kludged it back into operation (a) at reduced power and (b) in legacy mode.
Quote from: atop8918 on September 19, 2013, 03:53:33 AM
I'm not sure what is happening then. I wonder if RossW or Tom could chime in if they've seen similar issues?
Wish I could offer insight. I have no idea what that bunch of data means without knowing what it represents? I left my crystal ball in my other pants.
When mine locks up comms it simply refuses any connections. Ping gets replies but no access otherwise. It doesn't do it X times, its locked until reboot.
I will say it again in case folks missed it . I have one that loses comms regularly, say about every 24 hours and one that has done it before but has not for a very long time. The units are the same firmware but the one that is apparently fine has a SN that is +6000 from the one that is a problem.
Sorry I have no clue on what this data means.
Tom
The table i posted above, the final coumns "code" is the return value from newmodbus. 0 for ok, 1 for any positive value indicating a failed connect. Here itll drop occasional conects before closing completely.
Ross, do you have a list of return values for newmodbus?
As to why yesterday was worse, the only thing that i can think of was that ive had other network traffic running through the main switch, not heavy, but ongoing.
Ive been meaning to add a second network switch closer to tthe classic, so ill try that soon as well.
Ross, your pattern of non problems matches when i replicate this whole setup on a bigger computer. I had assumed yours was also a "bigger computer" but if its 1.2G maybe not much so, what is it, how many cores etc. Also, are you running it the same way as Tom, newmodbus -wi ? Youre interval is longer if 5mins. Its something ive pondered trying some other cron intevals lke 5mins. And i guess trying andrews method of keeping the connetion open. Other than that im out ideas as well.
Bob mentioned something on the resets thread yesterday about another new firmware coming.
Update, i added the extra switch, reduced the cron interval to 2 mins and took off the extra network load, and it hasnt lost a connect since. So one of, or all three of those factors are involved.
load average is now 0.13. Ill let it run for the day, then tomorow undo each of the three changes, one at a time.
Quote from: zoneblue on September 19, 2013, 03:29:55 PM
Ross, do you have a list of return values for newmodbus?
It always returns either a 0, or a 1.
It will return a 1 for anything I considered to be a "bus-related error".
Failed to connect, failed to resolve a host, data length error, modbus command response error, write error, etc.
It will return a 0 any other time - displaying a help message, normal results, required parameter missing etc.
Quote
Ross, your pattern of non problems matches when i replicate this whole setup on a bigger computer. I had assumed yours was also a "bigger computer" but if its 1.2G maybe not much so, what is it, how many cores etc.
FreeBSD 8.1-RELEASE #0: Sat Sep 18 17:20:47 EST 2010
CPU: VIA Esther processor 1200MHz (1200.01-MHz 686-class CPU)
Origin = "CentaurHauls" Id = 0x6a9 Family = 6 Model = a Stepping = 9
Features=0xa7c9baff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,APIC,SEP,MTRR,PGE,CMOV,PAT,CLFLUSH,ACPI,MMX,FXSR,SSE,SSE2,TM,PBE>
Features2=0x181<SSE3,EST,TM2>
VIA Padlock Features=0x3fcc<RNG,AES,AES-CTR,SHA1,SHA256,RSA>
real memory = 1073741824 (1024 MB)
It's a single-core device. Basically a carputer that I've got running as my home gateway/firewall/file-store/etc
Quote
Also, are you running it the same way as Tom, newmodbus -wi ? Youre interval is longer if 5mins.
# grep class /etc/crontab
# Get data from classic
*/5 * * * * rossw ~rossw/classic.sh
and classic.sh contains:
#! /bin/sh
# Address is captured from broadcasts by the classic.
# Run this:
# tcpdump -nl udp and port 4626|awk '{sub("\.[0-9]*$","",$3); print $3 > "classic.addr"; close("classic.addr")}' &
./newmodbus -w `cat classic.addr` 4115-4118/10 4119 4121-4122/10 4125 4132-4134/10 4120\>8 4275 4131 > classicdata.tmp
if [ -s classicdata.tmp ]; then mv classicdata.tmp classicdata; fi
wdt=`grep "^4131 " classicdata | /usr/local/bin/gawk '{print and($2, 128)}'`
if [ $wdt -eq 128 ]; then
mail -s "Midnight watchdog reset $$" (page-me-email-address) < classicdata
./newmodbus `cat classic.addr` 4131=12800
fi
Ross, interesting, so basically the same way im calling it. Interesting also on the baord. VIA x86. I have a really old one of those laying around here someplace a 400Mhz eden model. Might try it. At this point trying to narrow the factors involved. This might tell us something about x86 v arm.
Tom, heres something for you to try . Increase the cron interval from one minute to two minutes. Ive found thats made a huge difference. Only one dropped connection yesterday. Be interested to know what happens on the rPi. Also watch out for any single refused connections, as opposed to the ethernet lockup.
Some possibly pertinent system details:
Info for both units.
Web Access is disabled (mymidnite)
Both use port :502
Classic 150V (rev 4)
Firmware:
- Classic Rev: 1401
MNGP rev 1370 04/08/2013
Solar is #1871
Wind is #7805
I have some more data on the communication lockouts I am getting:
These are all the Solar Classic locking out communication.
##########
2013-09-17 03:14:
Solar stopped logging. No solar in, battery at 25.5V or so.
Middle of the night so no heavy network activity. Reset ~6:30AM with power cycle.
2013-09-17 unknown time.
no data gathered.
2013-09-18 16:34:
Lockout during low power FLOAT/EQ no direct sun, some bursts from wind.
Volts about 28.8 / 29
2013-09-20 21:08
No solar in batteries ~24.5
#######
All of these required a power cycle to get the Classic back online. Apparently still functions fine as a controller.
Has anyone tried using a different port? I have not. Might be worth a try. Sure can't hurt.
Both Classic 150's are on the same 4 port switch which is directly connected to a Dlink router and wireless AP. The Solar Classic has a short cat5 to the switch. The Wind Classic has a longer cat5 cable to the switch.
My gut feeling here is that there is an issue with my Solar Classic just not playing well with the logging App because the Wind Classic is fine. The recent data I collected while limited seems to disprove one of my older theories that it was related to high power throughput somehow. Most occurred during low or no power times.
I think Ryan is going to swap a Classic he has been logging from for mine to see if he can reproduce the fault.
Just an update.
Ok, I switched ports on both of them to :555. Edited and recompiled newmodbus for port 555.
Note, I could not do this with the Local App. Well, there is an option to do it but it just does not take. I did it on the MNGP "Misc" menu.
See if it changes the behaviour.
Just another change to see what happens.
Tom
Tom, I think I ~may~ have a fix for Classics that stop communicating and you have to reset the
Classic to get it back up and online. ~maybe~... Hopefully !!
There are a couple of little things to add regarding the Whizbang Junior and I would like to
also include those. This should be just a couple of days. I would be very interested to
see if this beta code fixes your lock-up problem.
boB
Quote from: boB on September 21, 2013, 03:18:42 PM
I would be very interested to
see if this beta code fixes your lock-up problem.
boB
boB;
Glad to give it a whirl. I just today reset both to use port 555 as it was one thing nobody mentioned and easy to try. So far so good after a few hours.
Running fine so far but typically I get a reset every 24 hours or so.
Send me a link or attachment and I will stick it on the problem Classic.
Tom
Quote from: TomW on September 21, 2013, 03:28:27 PM
Quote from: boB on September 21, 2013, 03:18:42 PM
I would be very interested to
see if this beta code fixes your lock-up problem.
boB
boB;
Glad to give it a whirl. I just today reset both to use port 555 as it was one thing nobody mentioned and easy to try. So far so good after a few hours.
Running fine so far but typically I get a reset every 24 hours or so.
Send me a link or attachment and I will stick it on the problem Classic.
Tom
OK, so not fully knowing everything I understand about all of this, it MAY be also that
this fix MAY have something to do with the resets. I just now realized that possibility
but am yet not quite assured of it.
I know you believe you understand what you think I said but I'm not sure you realize
that what you heard is not necessarily what I meant. Or something like that...
boB
Quote from: boB on September 21, 2013, 03:35:17 PM
OK, so not fully knowing everything I understand about all of this, it MAY be also that
this fix MAY have something to do with the resets. I just now realized that possibility
but am yet not quite assured of it.
I know you believe you understand what you think I said but I'm not sure you realize
that what you heard is not necessarily what I meant. Or something like that...
boB
Absitively, posolutely!
Never meant to imply I understood what you were mumbling about. :o
Tom
You know how i (unscientifically) changed three things at once. Will ive now isolated those by progressively undoing the changes. Increasing the sample period from one min to 2 mins was the thing that helped. Today there have been no lockouts, and the refused connections down to less than 2 per day.
Bob, good news on the firmware. Ideally i want to get the sample rate down to about 15 seconds, while still allowing the local app in. Also looking forward to the WB jr. So Blackbox can tell the complete story of production and consumption, exciting.
Tom: which leads to think that its the close proximty of your two connection requests that is possibly your issue. have you tried reversing the order that you call them to see if the problem child swaps or not?
Quote from: zoneblue on September 22, 2013, 01:53:41 AM
Tom: which leads to think that its the close proximty of your two connection requests that is possibly your issue. have you tried reversing the order that you call them to see if the problem child swaps or not?
blue;
Switching ports to 555 has gotten me a good 24 + hour run of no lockouts.
I will let this run awhile and if / when I get another lock out I will try switching the order or inserting a sleep delay in the script. One thing at a time. Currently it polls the wind unit first and that one has no issues (lately) so there could be something in the order / proximity of the poll requests.
If it turns out to be the port it will be a palm to forehead moment of why didn't I try that simple change before? If it keeps chugging along with the port change I will swap it back to an rPi and check it on that.
Just stumbling along early on a Sunday.
Tom
Well, got a lock out about noon so swapping the port didn't help.
Switched the order in which they run and left the port alone. See where it goes. If it changes which one that locks me out I will add a sleep delay in the script.
Just an update in case anyone is watching.
Tom
Having slept on this theres one other possibly significant difference betw our 'works' cases and 'not works': the arm boards typically have flash mass storage. Writes to common kinds of flash are slow.
I think my old 400Mhz VIA board will be a good test. From memory it has a 2.5 inch hard drive in it, with debian etch or thereabouts. Ive also ordered a faster uSD card to try. Lastly cubie has a sata port, and that coulld be tested.
Alas its monday again, at least here.
zoneblue;
I had a lock out today on the Solar (same Classic as before I switched polling order), so it is apparently NOT the order it polls them.
At a loss what to try next?
Just FYI.
Tom
You try 2 minute intervals? Guess we wait for the next firmware.
Cubie is still going, no lockouts for several days now. And its rendering graphs, doing database querys with several 10s of K records in the table, serving pages to sometimes as many as four computers.
Not bad for a wee lil board drawing little more than 1W !
I spent a while on the outback forum yesterday seeing what those guys are doing in terms of monitoring. That forum is pretty much dominated by a certain pair of commercial operators. Not having any outback gear im gona need some help to write a black box module for the mate 3. Its probably a ways off yet, as the focus is to get a 0.3 zip release out sometime.
Guys there is a possible firmware issue that we may have fixed. We should have new code today or tomorrow with that fix and the current sense module code as well as Days between Bulk charges. Stay tuned.
Ryan
This sounds great Ryan. Thanks, Vic
I have added support for multiple views, and an example here:
http://www.zoneblue.org/blackbox/index.php?id_view=2
My artistic ability is not that flash but anyone who can make an html page can make anything they want. The basic concept is that a view template has a number of 'Panes' and you can put any datapoint in any pane.
Im still waiting for the modbus-fix firmware, once thats in, ill make a zip. Meantime if you use SVN you can check out the code at http://code.google.com/p/theblackboxproject/ It runs stably enough if you limit the sample time to 2 minutes.
The next tasks are:
- to make it more flash friendly
- add better support for multiple devices of the same type.
- along with that will no doubt come hybrid datapoints that combine data from multiple devices, eg total energy today from all devices.
If anyone needs some help trying it out, im happy to help as it will provide insight into where the documentation (minimal so far) is weak. Also ideas for future development.
And oh if i could get one of the whiz bang things....
So is there a Software for Win 7/// I love this Interface.
With a fair bit of messing about it probably could be made to run on windows, but that isnt the point of it. Its designed to run on a network 24/7 so that it can collect complete data, and serve that information to computers and hand helds. If the computer isnt always on, there will be holes in the data.
The actual cost in running blackbox on its own computer is, what, about US$60 for the board, and a couple of hours installing the OS and other software. The latter is probably the dealbreaker for most windows folk, but we will hopefully be able to do some prebuilt micro sd card images and such later on. All you need to do is write the image to the card, stick it in and itll go. Well thats the plan.
I have a spare motherboard, I could run win Xp or even win 98 If I had to..
Quote from: SolarVet on October 09, 2013, 04:27:35 PM
I have a spare motherboard, I could run win Xp or even win 98 If I had to..
None of the participants of the "black box project" are currently inclined to write for windoze.
The whole premise of it is "lean and mean". Very low power draw, low hardware requirements. Not a power and resource hog like windows. Of course, once the "black box" is on the network and monitoring things, you would be able to view the details from your doze box (or indeed, anything with a standard browser).
Have been reading and following with interest the opportunity and diversion debate in NWAS forum
(where I have also posted similar to below)
and some very creative ideas for how to better harness and manage opportunity.
After reading as many posts as I can find, I would summarise that much of the effort is focussed on using opportunity
for "variable load heating" of one form or another - with various implementations using PV, DC or AC inverter loads.
Which no doubt works great for those in cooler climates ...
However my scenario is somewhat the reverse, and with some other less common characteristics.
I have no reqt for "heating water or similar" and hence can't use this variable load method.
But I have numerous "opportunity" loads that can be used in a combination over time to to simulate "variable"
Anyway to my point .... I've been speculating about a method to Measure the solar potential energy "solar irradiation"
and then use it as an "input" to a decision making process for "opportunity" load control
(For loads that cannot use the PWM methodology .... Hence the need for more informed decision Making)
Which brings me to the "blackboxproject" (BBP) - of which Im very excited (given my passion for home automation)
Here we have a fantastic opportunity to bring various inputs into a BBP that will already have access to much of the
data needed, and use it to make (or assist) informed decisions that can be used for load control.
Note this is not in anyway a plan to modify, interfere with or substitute for the core CC functionality it would simply
"wrap around" the Existing CC/inverter and Other components... And leverage the work and inputs already underway in BBP.
This would enable
- graph/charting etc of additional variables
- help understand better the RE system status and capability
- fault finding/preventative Maintenance (with future expanded sensors)
- better decision making for load management
- better data for planning
- some limited decision making and control capability
Required/suggested BBP sensor/inputs:
1) battery current/direction via current shunt
Q: can BBP get this data from CC via the modbus interface? (When CC fitted with the whizbang)
Hopefully yes soon (if not, need to add yet another voltage sensor across the shunt
To also measure battery current - direction AND value)
2) PV array
Voltage : BBP "vpv" - yes; Current : BBP "ipv" - yes
3) CC output
Voltage and current BBP - yes
4) solar panel temp
One or more temp sensors (1wire) mounted on rear? Surface of solar Panels
- measure/prove the temp vs power/performance Relationship
- relate to measured array voltage and Vmp (temp compensated from panel Temp vs voltage curve)
5) solar radiation (aka potential PV RE)
method 1 : solar radiation sensor
What type? How representative of potential power from real world panels? How to adjust for MPPT?
How does this allow for panel tilt ? And sun orientation? (Perhaps a shield to simulate panel orientation?)
It strikes me that with a database (collected over time.under actual operating conditions) of actual power
produced by CC when under "full load" ... And comparing this to the radiation sensor, it would be possible
to determine some algorithm that can use radiation sensor to estimate instantaneous production potential.
method 2a : use an actual solar panel (separate panel - not part of production array) with a permanent
dump load and measure instantaneous power produced "solar potential"
- a small cheaper panel could be used
- no battery required
- to be representative it would need MPPT capability?
Even with minimal panel, Charge controller and load, After adding all costs ... Would this be worthwhile... Hmmm
OR
method 2b : Lets find a very cheap way to do it - accuracy may suffer (could Be negated by the same method above to calibrate)
- Use a very small panel (eg 6V 2W) as "measurement array", permanently Fully load it (resistor load),
use a method For keeping it in a crude MPPT like region of operation ..(Research on this underway) and Measure actual power produced
Q: How representative could this be?
Summary : using all available data (much already avail from CC via BBP) and augmented with additional sensors,
Use real world measurements (charted over time) of "production array" vs "measurement array" to calibrate the relationship
and then use it to predict/ measure in real time, available energy - to intelligently switch Opportunistic (fixed or variable) loads,
And of course have a much greater insight into our solar PV system operation
Apologies that my first post is a big one ... but have been thinking this thru and researching how to get RE system data into
my existing HA system .... and finally was ready to put my (lengthy) thoughts into print.
To achieve my suggestion - requires a number of additional sensor inputs into BBP - but this is easily achievable using
arduino or RPi shields and readily available low cost sensors
(I personally run HA application on Win7 environment - but note and respect the plan for BPPto be Linux based on a Rpi or similar)
Thoughts from the BBP team....
Gregy, I have similar needs to yours. "oportunity heating" isn't of any interest.
Amongst my other meterological monitoring equipment however, I DID install an Apogee pyranometer. These are available as bare units, or amplified with a 0-5V or 0-10V output.
They are cosine corrected and are designed to be mounted horizontally, however for the purposes you and I want, mounting it in the same plane as your array is absolutely perfect. Its output will tell you exactly how much solar radiation is available to you. (My circumstance is complicated by having 4 tracking and 2 fixed arrays).
I use a CAI WebControl board to monitor ambient temperature, the pyranometer and various other inputs. It would be trivial to add a temp sensor (it uses the DS18B20 sensors) to monitor the temperature of one array for direct calculation of available power based on panel temp + solar radiation.
I won't post more here incase it's not of interest, but if it is... just ask specific questions for more info.
Hi Ross
If I recall from reading various posts - you may be a fellow countryman - albeit I don't live down under at present - indeed where I currently live, we use water from storage tank which is just fine for showering... No heating read
I don't want to hijack this forum unecesarily .... Maybe we should direct communicate or start another thread?
I use Homeseer (HS) HA pkg and have various sensor input capability,
I've seen the CAI device - however it's analog input (3 only) is way too limited for what I have in mind.
I've been anguishing over how
To get PV system data into HS And after much research found this BBP forum.
(Would be happy to separately share my research to date)
BBP will already have a lot of heavy lifting "done" eg getting access to modbus data and hence it's
One less thing to do, to achieve my desired outcome. Hence my suggestions above to enhance BBP
Edit - have sent you PM
Happy to start another thread, but also happy enough I think, for it in here as it is relevant.
The old version of the CAI device only has 3 0-10V inputs, plus another 0-5V used by a humidity sensor.
The new version (I have a beta one here) increases that to 8 analog, plus 16 digital in and 16 digital out, plus 16 temperature, plus a few other bits. Its the same formfactor, similar power consumption, same interface.
Additionally, you can read values from it by http, or it can push values to something else via http.
It is straightforward for example (as I have), to have it running code in the background to constantly monitor the maximum solar input, and the cumulative watt-seconds over a reporting interval (I use 5 minutes) and either push that to another device, or have something poll for it as required.
The aussie ripoff merchants wanted almost a thousand bucks for the pyro, I bought mine directly from the manufacturer for around $300, so it's not "trivial", but it is a proper scientific, calibrated, instrument.
If you didn't want something so "pro", it would be very straightforward to use a small cell (perhaps from a scrapped garden light) and an opamp. Mixing it with something like http://webcontrol.rossw.net/ makes a fairly neat solution.
Newer more capable version of CAI - very interesting!
Current CAI already has support for xaP, as does HS - hence I can easily communicate between HS and CAI (Aka heavy lifting done) and thus read it's analog and digital variables into HS And also send commands to control CAI digital IO
My planned alternate was to use Arduino (for which an Interface capability with HS exists) and
Leverage it's low cost, extensive Io capability (also easy to add low cost Wireless ) and range of low
Cost sensors.
Ouch that pyranometer is not cheap - get what you pay for I guess - as it's been prof calibrated.
I can see your dilemma with multiple fixed and tracked panels ..... You don't want to be buying more than one pyranometer !
I'm searching US sites on line ... But looks like a $300+ device ...
Your suggestion of using a cheap panel and opamp is along the lines of my above suggestion
However my proposed method has the advantage that if we can measure production array output (under full load) AND "test array" (aka low cost cheap solar panel) and chart them over a period
Of time and environmental conditions - we should be able to develop a reliable algorithm to
Use the "test array" to predict solar energy
I guess this is just a different way around using a $$$$ pyranometer - however my proposed method
Holds true either way - run a calibration period using data from production array to "calibrate"
The pyranometer or cheap test array
Either way we need access to all the data from CC ... Hence what led me to BPP - as it will access
CC via modbus and have access to all? The CC data
So using CAI or any other method to read sensors is fine, but we end up with data in multiple devices/applications
... So how to bring it all together is the BIG question
I've looked at using various modbus applications avail for win OS (my. HA environment) .. Which is possible, but there
Is a lot of work to interpret and translate the various modbus registers - as evidenced by the
Great efforts in this forum ... Basically I don't want to re-invent where it's possible to leverage
Existing capability
My alternate - is to work out how to get the data out of BBP (different Device and OS - I have no Linux experience) across a TCP/IP interface .... But this seems like reinventing also !
Would like to share some other thoughts on PV system automation and sensors with you - but don't
Want to further hijack this thread .. Untill we determine desire/capacity for BBP to be expanded to inc these
Additional inputs/sensors
RossW, Gregy, I have a question about the need to be measuring solar insolation:
Doesn't the CC do a similar function by 'determining ' the sweet spot for maximizing the array output?
I can see this if the array was tracking, but is there much benefit if it's fixed?
Tom I just noticed that the BBP is not a 'child' of the Open Source Software Section.
Can it be made one? and if there is a lot of feedback on this 'monitoring insulation' then make this a child of the BBP?
Quote from: Westbranch on October 17, 2013, 12:26:17 PM
Tom I just noticed that the BBP is not a 'child' of the Open Source Software Section.
Can it be made one? and if there is a lot of feedback on this 'monitoring insulation' then make this a child of the BBP?
Westbranch;
Not sure what you mean?
it is already beneath The Open Source software/hardware corner:
General info>> The blackbox project
And, not to nitpick but it is solar "insolation"
Tom
Hi Tom, duh! ya I know about solar insOlation, had to study that plenty.
Just didn't re-read before hitting SEND... will fix it.
I'll try to explain my thinking:
The Open Source Section has Beagle Board, Arduino and rPi sections and I see the BBP as another Software/Hardware type as it looks like it is/has become a separate entity by evolution.
Ciao
My understanding (and I admit im far from knowledgable on this)
Is that the MPPT in the CC will always maximize against the available power given the current
Solar radiation - but only when it "needs" all the avail power for the given charge state
Eg when in bulk it's trying to push max power into battery, but when it goes into absorb and
Particularly float ... It's not in MPPT mode - because it simply doesn't need all the power.
(This is a very simplistic of a much better explanation that I've read from bob)
So unless a load is applied, it's difficult (impossible?) for CC to determine how much available energy
Could be avail - because float (for example) uses only a very small ammount
Of potential solar capacity
Hence where the PWM opportunity load mode of operation becomes useful ... To slowly ramp up (or down) the load to find the sweet spot .... But it needs a load applied to do this, and by using a
Variable,load (controlled by PWM) it can (using feedback mechanism) vary to optimise for avail energy
So "our" (Ross and Greg .... No doubt others) dilemma is that we have "no use" for variable load
Heating ... And hence we dont have a (useful) variable load we can apply via PWM
And we are looking for a way to "forecast" potential nett excess energy (that's not being used for charging) and then switch on an appropriate combo of known, fixed value Discretionary loads
- without impacting battery cycle/charge state
Hopefully my poor explanation may make sense
Westbranch;
I don't seem to have the power to add a section? I will tag Ryan on this see if he can do it.
And, I know you knew what insolation is but insulation crept in during the discussion.
Future searches will work best with proper terms.
No offense intended.
Tom
Hi tom
Also pls guide as to whether my posts are appropriate for this thread .. Or better elsewhere
BR
G
Quote from: Gregy on October 17, 2013, 01:07:20 PM
Hi tom
Also pls guide as to whether my posts are appropriate for this thread .. Or better elsewhere
BR
G
Gregy;
Your previous post would probably have been better over in the Classic charge controller board:
http://midniteforum.com/index.php?board=2.0 (http://midniteforum.com/index.php?board=2.0)
It may be a bit off the topic in this thread. But we don't get too excited about that kind of thing.
Tom
Quote from: Gregy on October 17, 2013, 01:01:45 PM
So unless a load is applied, it's difficult (impossible?) for CC to determine how much available energy
...
So "our" (Ross and Greg .... No doubt others) dilemma is that we have "no use" for variable load
Heating ... And hence we dont have a (useful) variable load we can apply via PWM
And we are looking for a way to "forecast" potential nett excess energy (that's not being used for charging) and then switch on an appropriate combo of known, fixed value Discretionary loads
- without impacting battery cycle/charge state
So how do you envisage this 'forecast' process/calculation gets data to provide meaningful results?
The potential maximum available power should be relatively easily calculated and adjusted using actual historical data. However, the biggie issue here is the highly variable solar insolation and how to get real time data on this - obviously without loading the array and causing that impact on battery charge cycle.
Perhaps a separate tiny pv to use as a loadable indication of available power in main array?
Dgd
Greg, that is a good explanation, at least for me.. What non PWM options are you looking at?
Tom, none taken, my typing skills need a lot of improvement, as well as proofreading.. tks. Have to watch that spell check too
DGD
Yes - that's one method
Pls see my post #37 above - item 5) explains my proposed options (and yes one option is the "tiny PV array) - for which I have a proposal (Ross and I have separately been exchanging ideas)
Happy to post here also if there is interest
Westbranch
.. I'm glad you asked :)
Gregy's Diversion opportunity definition/categories
Fixed (known) loads for variable periods
DC load (at battery voltage) or AC load using inverter
- Cat 1 Opportunity : time shifted fixed loads
Can Simulate a variable load (but over a period of time)
Eg water storeage pump, desalination system, Electric vehicle battery charge, ironing :)
- Cat 2 opportunity : discretionary fixed loads
High pressure washer, boat wash, irrigation, A/C (wife may disagree)
2) variable power opportunity loads
Eg hot water storage, heating, fans
Can be controlled using PWM output with SSR
Works well for loads that can work with "variable power input"
I guess the above starts to give away my intended scenario :)
Think ... Surrounded by water, with a lot of sun shining ... definitely "way off grid" !
DGD
"Low cost Solar test array for measuring PV potential"
http://www.adafruit.com/products/390
Above is the link for circuit for connecting to a cheap 6v 2w solar panel
It's got a lot of stuff not needed for a "test array" (but it's all built to go at $18)
If you read the accompanying tutorial and in particular the "design items"
it seems they have done an admirable job to
Emulate a max power type of operation - so it's "MPPT like" for loading the solar panel
I was proposing to simply put a load resistor (in place of a LiPo ) and up the load current
Resistor control - So panel was always max loaded
And then simply measure voltage To calculate power
(May need a resistor voltage divider to keep ADC within 5V or 3V range )
So for around $18 (Adafruit board) + $12 6V 2W panel (eBay) = $40
Could deliver a good "test array" that operates in a max power style operation
And hence be a low cost predictor of available power once calibrated against the
Production array power measurements
..... $40 beats a pyranometer at $300
Further food for thought (yeah I know I have way too much thinking time)
If above works - I was considering using a second "test array" at different tilt (optimum
Seasonal tilt)
To determine opportunity benefit/loss of fixed tilt ..
. But this could also be Extended to tracking vs non tracking !!
(Projects for when we have idle time! Hmmm )
Gregy,
Thanks for link, excellent info there, looks like I will be busy constructing this sensor
Although my main diversion interest is water heating I have used a simple non pwm method using an external processor to simply divert output from a section of my solar array. Nothing too intelligent about the processing, the 'optimal' settings to ensure non interference with the battery charging process were arrived at by a trial and error process.
However, on good days a lot of my pv power is still wasted.
Dgd
Others have pointed out, elsewhere, that a single Photodiode with a heavy load, measuring its Isc, will be a good indicator of Insolation. Ideally, it (or any other such home-brewed detector) should have a diffuser. This is probably the tricky part. FWIW. Vic
It's getting late for me for sure ..
18+12=30 (not 40) ... Before anyone points it out!
DGD
At present I'm "workshop challenged" (long story ... Basically living in a hotel, no soldering iron ...
Getting withdrawl symptoms)
So I can't build this to prove it works .... Pls,let us know how you go.
Note however ... The key to making this work is the capability to chart "power" measurements
From test array vs production array (when both fully loaded) .. To calibrate and develop algorithm (as I'm guessing it
Will not be linear relationship ... That would be too easy)
Hence the interest in BPP ... Because that would make it's oh so much easier to capture and calibrate...
And then forecast and control
(Bringing us back to the thread topic !)
Vic
Yes your correct - diffuser required to address below "cosine" matter
I've seen various diffuser solutions
Ping pong ball being one of the more popular! .. I guess you just replace them as they
Deteriorate from UV, bird attack et al
And mount the sensor collocated at and orientated per the array (tilt and direction)
Quote clipped from
http://www.instesre.org/construction/pyranometer/pyranometer.htm
"important design consideration for even the simplest pyranometer. Ideally, pyranometer detectors should respond to direct sunlight in proportion to the cosine of the zenith angle of the sun. When the sun is directly overhead, define the "normalized" response as 1. As the zenith angle increases, the normalized detector response to direct sunlight should decrease as the cosine of the zenith angle. Real detectors do not have perfect cosine response. To improve their response, some kind of sunlight diffuser can be placed over the detector. Teflon® is often used because it has good spectral transmission properties, is very stable, and is relatively unaffected by long-term exposure to sunlight"
Given that calibrated high quality pyranometer can be $250+
I was searching for a lower cost method that achieved the stated intent ... Which will
Not give W/m2 absolute measure of insolation (pyranometer )
But will enable a forecast that can be confidently related to each systems individual production array
Similarly the photodiode light sensor options could also be measured and calibrated against production
Array ... We are not looking for absolute insolation measure, but rather relative measure of potential
We should try it also! If it works reliably with desired accuracy and it's cheaper solution - I'm all
For it
Hi Gregy,
Thanks for the additional info. Some day I will try to progress with instrumentation here, will SIT and wait for all of you folks to do the heavy lifting.
Thanks, Have Fun, Vic
Quote from: dgd on October 17, 2013, 01:33:43 PM
So how do you envisage this 'forecast' process/calculation gets data to provide meaningful results?
The potential maximum available power should be relatively easily calculated and adjusted using actual historical data. However, the biggie issue here is the highly variable solar insolation and how to get real time data on this - obviously without loading the array and causing that impact on battery charge cycle.
Perhaps a separate tiny pv to use as a loadable indication of available power in main array?
OK. A few years ago, I installed the pyranometer at my own site so I could measure the actual power available at any given time. ONE of my purposes for this was to calculate the actual efficiency of my entire system. By measuring the power from the arrays vs what was actually available tells me how well I'm collecting power. By measuring how much my charge controllers delivered into my batteries tells me their efficiency, etc.
HISTORICAL INFORMATION IS USELESS here because it can't predict what atmospheric conditions we have (cloud, smoke, dust etc).
I also calculate every minute, where the sun should be and from that what the radiation should be. By comparing that to the pyro output, I am detecting (quite accurately) small amounts of "lost radiation" due to high cloud, haze, smoke etc.
Additionally, by knowing the radiation present, I am able to calculate a "predicted power" from my arrays, knowing where the sun is (by calculation) and how much sun there really is (from the pyro) and calculating the error angle (absolute off-axis vector elevation and azimuth) from each array to the actual sun position.
Between all these, I've a pretty good idea of just how much power I *COULD* be getting from each array (and in total).
Now, here's the kicker (the bit I haven't done yet).
As explained elsewhere by others, the classic is only drawing as much power from the arrays as it needs to supply the batteries and instantaneous load. If I "could" be making 3KW but the batteries only need 400W (because they're in float) and the load is only taking 800W (because that's all that's running now), I have a "potential" of (3000 - 800 - 400) = 1800 watts of "potential power" that I could be using for something else.
Ideally, I should "switch on" about 1800 watts worth of opportunity loads.
It also needs to be "smart enough" to drop opportunity loads when we don't have enough headroom to maintain them all AND keep the battery and load supplied.
Such a rash of interest in the project. Cool.
Work's a bit crazy at present but ill try to catch up reading in the weekend.
Briefly. Insolation: how much power arent i using, but could be?
The latest trunk code has a solar insolation module built it. It calculates insolation from first principles, using a variety of algorithyms, the most promising of which are from pv-lib here:
http://pvpmc.org .Those are "clear sky" models. Ross's insolation measuring approach is better.
Secondly, i agree that the CAI is more or less perfect companion for the black box, such that ill be mounting at least one inside the 'box' here. My interest in co-collecting weather data is shared with you guys it seems. We may as well have one device powered up and storing everything.
I also have been reading about reference panels on the same plane as the array, and ordered a small glass covered mC panel from dx.com last week to test. From what i read about reference panels its the Isc that is of interest, so shunt, feeds adc is probably all you need.
More about pv modelling and monitoring here:
http%3A%2F%2Fenergy.sandia.gov%2Fwp%2Fwp-content%2Fgallery%2Fuploads%2FSAND-2004_PV-Performance-Array-Model.pdf
http%3A%2F%2Fprod.sandia.gov%2Ftechlib%2Faccess-control.cgi%2F2009%2F098258.pdf
the latter of which covers reference panels.
Lastly, what if we had 12+ adc channels. Then we could monitor each battery cell's voltage. Now THAT would be cool. Pick up imbalances before they get out of hand.
Ross. I havent hooked up the webcontrol yet, but i noticed in the manual, it says you can also add additional adcs using 1 wire, of which there are several ports on the board. Will that work? I also note that there are now arduinos with 12 bit adcs.
WB jr. You can bet BB will have support for this soon. I hope Bob will be willing to swap ideas on SOC modelling, and im particularly interested in also modelling the decay of battery capacity over time, similar to how linux laptop battery monitors work. With the extra cpu BB is ideally placed to do these sorts of computations.
Re using BB to control/manage. My focus for now is just monitoring, but once thats mature, i have little doubt that things like weather forecast lookup, CC and load managment will be candiates for future devleopment.
Now that we have our own sub forum (thanks!), i guess we can break this conversation up into threads. Phew.
Quote from: zoneblue on October 17, 2013, 05:15:43 PM
The latest trunk code has a solar insolation module built it. It calculates insolation from first principles, using a variety of algorithyms, the most promising of which are from pv-lib here:
http://pvpmc.org .Those are "clear sky" models. Ross's insolation measuring approach is better.
Tried working it out but nature has other ideas. (Part of my application, incidently, was to quantify the effectiveness or otherwise of my 60-evacuated-tube solar water heater). Knowing what was actually there is far more relevant than what some hypothetical calculation says might be, if I were at 30,000' !!
Quote
I also have been reading about reference panels on the same plane as the array, and ordered a small glass covered mC panel from dx.com last week to test. From what i read about reference panels its the Isc that is of interest, so shunt, feeds adc is probably all you need.
Yes. (If you also want to use it for meterological work - calculating evaporation rates, growing potential, etc, etc, then you do need a proper, cosine-corrected, horizontally-oriented sensor)
Quote
Lastly, what if we had 12+ adc channels. Then we could monitor each battery cell's voltage. Now THAT would be cool. Pick up imbalances before they get out of hand.
Greg and I were talking about that in PM last night. I've a base design I plan to develop shortly, its been sitting collecting dust for 12 months but I really need to do something about it!
Quote
Ross. I havent hooked up the webcontrol yet, but i noticed in the manual, it says you can also add additional adcs using 1 wire, of which there are several ports on the board. Will that work?
A qualified yes. I am using it to monitor my 6 arrays plus array voltage (3 original ADC channels + 4 on an external ADC). Maxim however discontinued the DS2450, so it's not an ideal thing to build into new systems. Against that, I've found someone who has used a PIC to "emulate" the DS2450, using its own ADC and simulating a 1-wire device.
R.
Quote from: RossW on October 17, 2013, 04:07:10 PM
OK. A few years ago, I installed the pyranometer at my own site so I could measure the actual power available at any given time. ONE of my purposes for this was to calculate the actual efficiency of my entire system. By measuring the power from the arrays vs what was actually available tells me how well I'm collecting power. By measuring how much my charge controllers delivered into my batteries tells me their efficiency, etc.
HISTORICAL INFORMATION IS USELESS here because it can't predict what atmospheric conditions we have (cloud, smoke, dust etc).
I also calculate every minute, where the sun should be and from that what the radiation should be. By comparing that to the pyro output, I am detecting (quite accurately) small amounts of "lost radiation" due to high cloud, haze, smoke etc.
Additionally, by knowing the radiation present, I am able to calculate a "predicted power" from my arrays, knowing where the sun is (by calculation) and how much sun there really is (from the pyro) and calculating the error angle (absolute off-axis vector elevation and azimuth) from each array to the actual sun position.
Between all these, I've a pretty good idea of just how much power I *COULD* be getting from each array (and in total).
i can see you have put in some serious effort to evaluate the solar potential. The only difference with my location in suburban west Auckland is that the amospheric conditions seem much less vairiable, negligible smoke and dust and minimal cloud - in fact I have recorded conditions over the last three years and have 85% clear sunny, minimal or no cloud conditions, in that time only four occasions where dense cloud cover and rain lasted longer than 3 days. The only disadvantage to this is that it takes an hour to get seriously burned by UVB.
This means I can depend on more stable predictions based on historical data...
Quote
Now, here's the kicker (the bit I haven't done yet).
As explained elsewhere by others, the classic is only drawing as much power from the arrays as it needs to supply the batteries and instantaneous load. If I "could" be making 3KW but the batteries only need 400W (because they're in float) and the load is only taking 800W (because that's all that's running now), I have a "potential" of (3000 - 800 - 400) = 1800 watts of "potential power" that I could be using for something else.
Yes I am well aware of this 'problem' as I see power waste all too often.
Just as an aside I recently decided to remake the mounting frames for my panels and after seeing the bank was fully charged I disconnected most of my panels while rebuilding frames at a different location (less shade problems). I kept 6 by 140watt panels in one string to a Classic250 connected. I was surprised to see that over 4 days this much reduced PV input still had the bank into float by midday and my plans to reduce load by turning off the beer fridge (and consuming contents!) was not necessary. My normal AC loads were much less than I expected and on reflection the process of replacing stuff with its lowest power equivalent (led lighting everywhere, 230Kwhr/year fridge/freezer, 229Kwhr/year LG 55 led TV, energy efficient washing machine etc.) had clearly worked!
Quote
Ideally, I should "switch on" about 1800 watts worth of opportunity loads.
It also needs to be "smart enough" to drop opportunity loads when we don't have enough headroom to maintain them all AND keep the battery and load supplied.
I do that for hot water. Now my rather simplistic HW controller just switches in half my PV array when bank goes to float and PV voltage rises sufficiently so that removing half PV array does not drop float. I currently use a eeepc that does nothing more complex than measure the PV voltage which rises when Classic demands less power to keep bank charging.
This simple method probably accounts for a large proportion of the possible 'excess' power but a more intelligent and accurate measuring method, black box based?, should allow greater use of the potential excess.
It would be nice if you and Gregy (and others) could keep your discussion here rather than by email as I, for one, would appreciate learning more about how to measure and get more efficient use of excess power.
dgd
(thanks TomW for moving this thread to a separate topic, I would have liked to see a separate water heating topic as after battery charging it seemed, to me anyway, that there was a lot of interest in using excess power to heat water BUT then again maybe not
as others have said here they want to use excess power for other purposes)
Ross, PVlib has a whole set of functions, which are well tested in the field, for taking the planometric insolation, typically from a meterological TMY file, but your plano is perfect, and takes you through a whole series of adjustments. AOI, Reflection and refraction off the glass, airmass color spectrum, cell temperature, iv curve etc, and arrives at a dc mmpt output voltage and current. Its quite cool. Its coded in matlab, which isnt overly onerous to port.
The alternative is to actually use a 'reference panel' of the same type and characteristics on the same plane, and actually measure its output. Both are worth pursuing imo.
DGD
Over my above posts and the below, I will have replicated all of the PM exchange between Ross and I
As I was new to the forum - and had a few possibly whacky thoughts - I wanted to bounce of someone
Before potentially embarrassing Myself ...but I see you are all a great bunch - so herewith all crazy
Thoughts will just appear regardless :)
... Now don't complain over size of my posts !
Zoneblue
I'm incredibly impressed by your coding effort and level of detail - I wish I could offer help on coding,
But found after one year at start my career that sw coding was not for me - these days I'm
High level only (a simple script is my limit)
I'm thinking through how I can contribute given various factors and location .....
To sensors
(Background - I've been using Dallas 1wire for 15+ years, and tried and used various others on home
Automation and weather collection system)
Temperature
** hard wired
DS18S20 / DS18B20 / DS1822
Hard to beat the Dallas 1 wire (provided that your HW has 1wire hub support - more later)
there are a few different models, thing to be careful is that some are parasitic (DS18S20) others not.
1wire sensors are exactly that, 1wire for data, and a ground (usually a second wire)
However some 1wire sensors require separate power, as the bus "parasitic" power is insufficent, this
Also becomes an issue as both wire lengths and number sensors increase (then it becomes effectively
2wire plus gnd)
the CAI will only support non parasitic (DS18B20/DS1822) max 8 - which slows down the CAI polling.
Analog and digital IO
1) DS2450 : analog input
was reported to be unreliable and susceptible to noise, volume consumption dropped
And as you correctly point out, maxim dropped it.
Agree we should steer clear of relying on this sensor.
2) CAI : I have not used it personally, it's (current model) 3analog inputs and 8 digital IO limits
Its potential as your system grows, and indeed in my system is way too small to begin with.
It has ethernet connectivity (good) and also has inbuilt web server (good) ... But of course must be
Connected to a hub/switch with Cat5 cable ... So if you want to meaure "remote" info,
It must be collocated (not good to locate the ADC inputs at any distance for the signal being sensed)
Because it's already an Ethernet device, working with it is quite easy AND it can operate standalone, and has some on board logic (PLC) like capability.
Summary : works well for local sensing or where your LAN goes, but it's (current) analog input (3) is
Too restrictive (note it's rumored there may be a newer version coming with greater IO)
3) Dallas 1wire digital IO
Various options and too many to discuss here, can be useful if 1wire bus already available
4) PICs / Arduinos / RPi (with shields)
Lots of different options available - these work well for low cost distributed nodes and for
Taking a lot of the low level sensing and calculations away from main Automation server
these are fast taking over - because of wide support, myriad of HW options, high density
Analog and digital IO, low cost, low PWR options, wireless support, ... Benefits are many.
As distributed sensing and IO nodes, IMHO - it's hard to beat where arduino is at.
Note options will differ widely of course, if you are experienced and competent in PIC, then why learn
Another etc etc
For me - I think arduino is best choice IMHO IMHO
5) arduino
Examples D= digital IO, A=analog Input. (Note also that some digital outputs have inbuilt PWM also)
UNO 14 D, 6 A. $30
Mega 54D, 16A. $48
Note the prices are for "official" versions, there are a bunch of 100% compatible Chinese versions
On ebay and elsewhere for starting at $18 for mega and $12 Uno
yes $18 for a device that has on board 54D 16 analog ADC capability
Breakout boards and sensor support is endless
AC Current via Non invasive CT, DC current with Hall effect (yes measure each of your DC loads up to 30A),, gas sensors, high voltage isolation, current shunt sensor (a whole diff topic)
Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!
(However see my following post on this specific topic - a summary of the Greg and Ross offline discussion)
Now to BBP (I'm still trying to be on thread topic)
Linux based - already has support for 1wire library, I2C (talking to arduino), SPI and many others
if it's RPi (I recall that rpi was planned as one of the supported BBP HW platforms)
there is no on board ADC - but can be easily added with shields and/or "talk to an Arduino"
for "local sensing" it may be hard to avoid the simplicity of just adding a shield to Rpi (there are
Many choices to add multi channel ADC, expand digital IO, 1wire bus) - indeed I've seen one
That will do all this on one shield
For remote and distributed sensing, IMHO hard to beat low cost low PWR arduino uno (@$12)
There are even arduino nodes with inbuilt wireless avail for as low as $20
DGD - you asked for the detail ....
Zoneblue - separate question: if RPi is a planned HW platform for BBP, what are the options/plans for memory storeage support ?
I forsee that as sensors grow, storing all the data may outgrow on board memory cards
(Given what I've seen - I'm sure you have this already thought through .... Just curious your plan)
Should we have a different sub thread for "BBP sensors/IO" ?
Zoneblue
One of my specific requirements is to be able to get access to BBP data into my separate home automation server
So I'm interested in how BPP would be accessed ( to read data ) from a LAN connected
PC
Noted your busy - so whenever you get a chance to respond - look fwd to hear further
Quote from: Gregy on October 18, 2013, 01:05:44 AM
Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!
(However see my following post on this specific topic - a summary of the Greg and Ross offline discussion)
I haven't done any more real thinking on this since we emailed last night Greg, but I did have one idea that I'll try to make some time to play with shortly.
Plan: a tiny, almost trivial, board using a PIC or PICAXE. 8-pin job. Has three analog inputs.
One device across THREE cells, can measure cell 1, 2 and 3, while being powered off nominally 6V.
Cell 1 directly into input 1 (about 2V).
Cell 2 through a 2:1 divider (about 2V)
Cell 3 through a 2:1 divider (about 3V)
This device reads the cell voltages and sends via serial "up the chain".
Next device - identical to the first.
Its "serial input" will now be flapping about between around -6 and -2V with respect to its own ground.
Simple resistive divider from second boards Vcc to input, and input to previous board, we should now have a sensible-enough input to read the serial.
It receives "Cell 1: xV", "Cell 2: xV", "Cell 3: xV", "End" via serial. As it receives each, it "passes it up the line".
When it gets "End", it adds its own three cells picking up from the last. (adds cells 4, 5, 6).
This ripples up the chain. Each little board does 6V, so 2 boards for 12V, 4 for 24V, 8 for 48V.
Each one has its own "local reference" so the problem of differential creep across 24 cells is reduced.
The last board will output its data straight to something to actually log/use. Perhaps by IR LED.
As I was typing, additional ideas sprang to mind.
Feed back the last board to the first. Now, every board knows what every OTHER cell is. It can compare its own cells voltage to the "across the entire pack" average and will know if a cell is out of balance - regardless of the SoC of the whole pack.
Add a LED. Now a board can indicate a problem cell in a stand-alone configuration. Walk out and immediately see. If board 2 is flashing its led two blinks, pause, two blinks, pause, it's its second cell that's crook.
Sure, monitoring, logging and alarm can be added, but something to show you immediately that there is a problem and which cell it is, a good thing!
I reckon I can make 'em on a sliver of a PCB that will be perhaps 3/4" x 1/2"
Will try to find time to lay some boards in the next week or two.
Anyone want to pre-order some? :) :)
Ross
I have a more detailed post under preparation (that covers various considerations - and the
Multi CH ADC challenges to zoneblue question)
However a few questions and observations
- what is stability/accuracy of the pic voltage reference?
- what is the value of Vref?
- what's the ADC resolution?
All the above will impact measuring resolution and accuracy.
Agree that by reducing the number cells, this minimizes the differential creep
- PIC. (And arduno) ADC requires the input impedance to be around 10k
So a 2:1 divider will put a 20k load across each cell (for 2V cell that's 0.1mA "permanent"load )
I think this is manageable .... Probably within the self discharge/leakage range ... And if batteries "always"
Connected to charger .. Not an issue ) as some batteries will not have the divider, there is a
Slight imbalance in load/discharge between cells, but again it's very low
- I'm trying to get my mind around what happens to resolution with a voltage divider
With a 2:1 divider it halves resolution?
thus
10bit ADC using 5Vref gives 4.9mV resolution, for each differential measurement (2 measurements) this reduces to 10mV (rounding to make it easier) .. And the the impact of voltage divider? Does it again halve?
And hence is it effectively 20mV (I'm reaching for paper and pencil! ohms law will be my friend)
Q: what resolution is needed to be able to Compare cells for "problems" ?
we may need to be able to measure at 20mV resolution (or better) to be able
To meaningfully measure differential cell voltages (for 2V cells) - if resolution or accuracy to low
Then the results may not be useful ... And could be misleading? (But at least would prompt a
SG test and a DMM cell measurement!!)
Edit
original post edited - removed reference to mfr referenced volt differential - as I think it was SG differential
That was being reffered to
I was planning to use a 3.3V (or close to) reference, because in a worst-case (single cell gone short circuit) I can only rely on around 4V to power the device.
Using 10-bit ADC would give nominally a 3.3mV/bit resolution. Of course, it isn't that good, so lets say <10mV/bit.
With oversampling we can crib some of that back.
With a 2:1 divider we're now back to around 20mV/bit.
Ways to overcome it. Easiest is to add a quad opamp, like MCP6004. Can level-shift rather than using a divider and back to 10mV/bit. If we're clever, level-shift AND add some gain. If we said 1.5 to 2.5V translates to 0-3.3V into the ADC, we have now 1V/2^10 bits, or near enough 1mV/bit over the reduced measuring range.
That also addresses the 20K load, but adds some cost, potentially some "calibration" required, and of course the device itself will be drawing some modest power (perhaps a milliamp?). The voltage reference/regulator will likely be the hog.
Agree that using 3.3Vref is better - helps improve resolution
The figures are for "each measurement" ref to module 0V ?
Hence when doing the subtraction to get differential (eg individual cell)
The effective resolution halves again?
I still think it's more than OK .... I've been doing some further research on the "allowable"
Differential voltages in 2V cells (not yet ready to put fwd a decision on this..)
But I think if we can be 20mv effective for a 2V cell (after all the halving et al) we have a useful
Measurement (if.nothing else at least use it as a trigger alert for "do a SG test" or further investigation required! )
.... But I really do want to find a way for us to capture this into a device for "automated alerting and monitoring" (visually checking a red led or similar is good ... When you regularly visit the battery bank,
But there will be requests no doubt for a more remote monitoring capability ... Me thinking aloud)
Quote from: Gregy on October 18, 2013, 04:13:59 AM
.... But I really do want to find a way for us to capture this into a device for "automated alerting and monitoring" (visually checking a red led or similar is good ... When you regularly visit the battery bank,
But there will be requests no doubt for a more remote monitoring capability ... Me thinking aloud)
Oh, the visual display was just an "added bonus". My intention was always for it to be externally monitored/logged.
The "end of the line" would operate an LED, which could be optically detected and fed into a conventional input on "something". Full isolation. Constantly updated.
What's the max allowable supply rail for the PIC?
Remember an EQ charge can raise per cell Volts to around 2.57V (depends on battery mfr recommendations)
Still within the 3.3vref range for ADC - ok
But you could see supply rail of 3x2.57=7.71VDC
Ross - I can't say I completely follow the design logic of the "serial up the chain" - you are way more creative Than me!
And certainly you have good PIC experience ... So I'm mute on that aspect of design
But the multiple nodes - with flexibility for different cell voltage/string composition sounds good
And one node for every three 2V. Cells should give resolution and accuracy and balances the various
Issues we have noted, also makes self powered viable (need to check the average battery drain
When operating)
Break out your breadboard, soldering iron and PIC programmer :)
Look fwd to hear the results
Thread moderator - I really think we need to break the threads up - as we are mixing up a bunch
Different issues and I don't want us to loose focus on BBP also
Perhaps a separate thread? Or place for "BBP sensors and input"
(Just a suggestion)
Zoneblue ... As promised - some things to consider for battery cell ADC
Measuring battery array cell voltages
The challenges
- each cell (except for bottom cell) has a negative terminal that is NOT at ground potential
When things are referenced/connected to ground - it's termed "common mode"
- measuring individual cells in a string requires differential or other techniques (common mode
Technique becomes a challenge as the number cells increases)
- cell terminal voltages need to be "isolated" from ground in measurement sensor
- sensing equipment should put minimal load onto the individual cells (we don't want to create
The very imbalance we are seeking to measure!) ... And we don't want to drain the batteries unecesarily
- need sufficient resolution and accuracy to be able to get useful data on cell imbalance
- typical ADC in low cost devices (arduino, PIC, CAI etc) can only measure 5V, for higher voltages
This requires use of a voltage divider
- voltage divider puts a load (albeit small) across the cell "permanently" (unless elaborate switching
Is used to disconnect when not measuring)
- to get high accuracy, each voltage divider needs to be accurately measured, or each ADC channel
Individually calibrated (digitally in sw) Against its respective voltage divider
- various schemes using relays are possible, but this method has challenges, not to mention lots
Of relay clicks and mechanical reliability
Common mode "subtractive method"
Measure each cell referenced to ground, and then subtract the next lower cell, to get the differential
Voltage of each cell (what I call the sequential subtractive method)
Works well for smaller number of cells, but as the number cells increases, the number ADC channels
Increases also
Eg for a 48V=24x2V requires a 24ch ADC .... In a single device almost Impossible to find
Easy you say ... Let's add a second device .... Now this is where common mode isolation bites you
Eg Arduino mega has a max 16ch ADC (works great for all but 48V using 2V cells)
When you add a second board/module - it MUST be electrically isolated from the first.
Isolation can be achieved via optical, wireless or electrical techniques
one method is to use a number of isolated modules and measures a few cells with each module
Ross has proposed a neat way of using IR led from each sensing module to isolate it. (The optical isolation technique)
Now here's design considerations when using the above multi Ch ADC method
- Each cell needs an individual voltage divider referenced to ground (aka common mode) to
Be able to use a 5V (or lower) ADC .. So each cell has a current draw, and there are rules for stability on the ADC inputs that require a certain input impedance, typically 10k ohm)
hence value of resistors in volt divider will change (as you go up the string)
- ADC with 10bit resolution allows 1024 steps between 0v and Vref, if using 5v reference, this gives
Resolution of 4.9mV (note I said resolution ... Not absolute measurement accuracy - which requires
Each ADC and input to be separately calibrated against high accuracy voltmeter)
Note accuracy is impacted by a lot of other factors (inc the Vref source)
- however the resolution halves when using the subtractive method (there are two ADC measurements
To determine each cells individual voltage)
So it can deliver 10mV resolution for each differential ( 5Vref) measurement
(See also the above various posts between Ross and Greg)
Dedicated measuring ICs
Various chip mfrs have solutions custom designed for the multi cell measurement
Environment ... But finding any commercially avail built up boards has yielded
Me no acceptable results to date
I've researched a few and one of the more promising is
maxim max14920/1 (12/16ch)
However to date no "commercial" solutions have been built.
Note there is advertised by maxim an "evaluation kit" ("max14921evkit")
That is fully built (supports 16ch) - but no pricing avail (could be a sticker
Shocker I suspect)
It has on board USB fully isolated (can stack 2 or more and avoid the common mode issue) and has win s/w driver and neat application for read and display cell voltages (perfect for win people,like me :)
Summary : for a BBP "battery string cell voltage measurement" at present the modular PIC/arduino type solution appears the best
(Eg per Ross proposal and the above various posts)
However it's achieved - I remain convinced its a useful additional input into BPP
And can be used for monitoring, optimizing and early fault/maintenance indications
Of cells in the battery string
... yet another set of sensors for BPP to store (thus need to think thru how much
And frequency of historical data to store!)
Quote from: Gregy on October 18, 2013, 01:05:44 AM
Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!
Would another approach be to use some kind of multiplexer? Gees you know how it is, so i have 12 cells, the next wit comes along has 24 cells... A multiplexor into a single adc might scale. Battery cells obviously dont need a lightning fast sample rate. Just thinking out loud.
Quote
One of my specific requirements is to be able to get access to BBP data into my separate home automation server So I'm interested in how BPP would be accessed ( to read data ) from a LAN connected PC
Thats the easy part. I envisaged a simple http API that can exchange data with other devices, in particular hosted servers that act as go betweens / backup / replication etc.
QuoteZoneblue - separate question: if RPi is a planned HW platform for BBP, what are the options/plans for memory storeage support ?
Well, obviously with linux a whole rainbow of options is open to you. You can replicate data to a NAS already on your network, push it to any number of online storage options, dropbox, s3, whatever. Im sure ross would help with some cheap hosting for those that want to go that route.
Now of course we need to bear in mind, at every step, our old friend power consumption. None of us wants to use a bunch of amps to measure how many amps were using. I have a bunch of other computers running part of the day for various purposes, but i only want that one device running 24/7, so its got to be sub 4W total. The cubie draws 1.25 so that leaves about 2.75W for the switch and the analog stuff.
A 32 GB SD card will store a fair bit of data. Lets see, the cubie's mysql database here is currently at 80KB for 75 days. At that rate, itll take... 1095 years to fill it. But for me that isnt really the issue, the issue is my low level of trust in sd cards. Most of the newer gen arm boards have sata ports and the eg samsung 840 series SSDs idle in the 50mW range. Does that give you a clue as to where im headed with mine?
All good discussion.
I looked at some capacitive multi-pole switches for another application. These move both sides of a cap to the "under test" circuit, then quickly switch both sides back to "local ground" to measure without the common-mode errors. It can work with several hundred volts differential, and it's remarkably accurate. But with it comes a price. And it would require one per cell.
The pic+opamps solution is currently looking like a good start.
Only 10 bits is a bit of a pain, but there are things we could do to increase that. Oversampling, switching a small bias and resampling, etc. All trickery, but all valid. At the end of the day, how much precision do we really need?
Accuracy can be provided by calibration, but precision needs to be built in.
In my most recent considerations, a closed loop where every block knows the average voltage of each cell in the entire bank, makes for a very good "something needs a human to look at me" alarm.
I will be interstate much of next week and have a power of stuff to get through this weekend before I go, so unlikely I'll get anything done on it in the immediate future, but I've put it in the "think about it and get back to me" part of my brain. It should start popping answers at me soon :)
Zoneblue
Thanks for response - good news to know that access to the BBP data will be "easy"
You started me reading up on rpi vs cubic vs beaglebone black
Wow I didn't realize there was such a difference
Certainly the last 2 are a lot more capable ... Price points similar (frankly not a consideration, there
Are other more important considerations for the task at hand)
Both have much greater IO as c/w Pi ... Hence making "on board" IO a definite possibility if using
Beaglebone or cubie
Eg beaglebone black has 7 ch 12 bit ADC (but only 1.8V max .. Noted by some as RPITA)
Sounds like you have already settled on cubie (perhaps that was a timing issue)
So now I'm rethinking my plan - seems cubie or beaglebone black are the current better solutions
For hosting a BBP
DGD
Do you have a link to a post where you describe details of how you are doing your panel isolation
And switchover to heating loads (I'm not thinking of heating)
But I'm speculating this may be a better way to achieve some other things I'm contemplating
Not clear if you are switching a complete controller (assuming you may have more than one) away from main bank (I'm assuming that's not possible because the CC. Are linked)
So if that's not the case, you are switching over at the PV panel high voltage - into ? Another CC?
Or somehow direct across the load ? (But then no MPPT operating )
Just curious ..... DC switching at 150V or higher .. That can get exciting I suspect - but if there is minimal load I guess it's not as bad
FYI, I came across the short video part way down this page, there may be some info that applies here
http://www.adafruit.com/products/1278
Just popped in for a coffee and thought I'd check on reference voltages and check more carefully on the later 08M2 parts analog.
Seems the picaxe now has access to an internal (quote) "Accurate 2.048 volt reference" for the ADC.
Using 0V/2.048V internal reference gives us immediately a 2mV precision.
If I use the opamps to reduce cell drain and purely as a level-shifter, I can get away with a dual rather than quad opamp (smaller, cheaper, even lower power) and maintain a 2V measuring span for each cell.
My only concern is how I accurately level-shift without a reference....
*ding* Blinding flash of light.... use the "top" of the previous cell! Ie, differential amps. Each one references its output on the negative of the cell under test!! 0-2V (nominal) out. I will still need a small bias or divider to be able to measure over 2.048V (really need to get to about 2.5V). Simple 4:3 divider with 1% resistors still leaves me around 3mV/bit resolution. If we're aiming for 10mV or better, we're comfortably in!
As for electronics in the battery box: they could be potted.
As for calibration: it could be done digitally and stored in the pics NV scratch. (I currently do this with another device I make for light aircraft)
Ross
I think we need to aim for per cell measurement of at least 2.7V (for the 2 V cell scenario)
for flooded lead acid cells eg surette rolls that they recommend
A EQ charge in range "61.92-64.08" (48V string)
Hence if all cells equal during EQ, This could already give 2.67Vpc
(Not sure if this temp compensates up or down)
And I'm thinking that during EQ is exactly one of the very useful times to have accurate individual
Cell measuring (To supplement SG testing of course)
Not sure about other battery chemistries .. Is the flooded lead acid battery our "worst case"? Design scenario
If we level shift to measure against the 2.048Vref. We can discard the lower end
Eg if cell less than 1.0V it's got much bigger issues, hence we can get 2V measuring range centred around 2V
Eg 1-3 Volt measuring range
Quote from: Gregy on October 19, 2013, 12:05:58 AM
Ross
I think we need to aim for per cell measurement of at least 2.7V (for the 2 V cell scenario)
Now it would be nice if the per cell measurement could range high enough to cover 3.2v Lithium Iron cells :)
dgd
Quote from: Gregy on October 18, 2013, 09:40:15 AM
DGD
Do you have a link to a post where you describe details of how you are doing your panel isolation
And switchover to heating loads (I'm not thinking of heating)
But I'm speculating this may be a better way to achieve some other things I'm contemplating
Not clear if you are switching a complete controller (assuming you may have more than one) away from main bank (I'm assuming that's not possible because the CC. Are linked)
So if that's not the case, you are switching over at the PV panel high voltage - into ? Another CC?
Or somehow direct across the load ? (But then no MPPT operating )
Just curious ..... DC switching at 150V or higher .. That can get exciting I suspect - but if there is minimal load I guess it's not as bad
I tried switching out two strings of panels (from 4) but it made no difference to the Classic. Its a 150 with 2.8Kw PV in 4strings by 5 panels.
24v bank, so too much PV input but designed to give some useful power input in non ideal conditions.
I just insert the heating element on the output of two strings and protect the Classic with a diode to stop risk of Classic feeding the heater.
Auctually use a 3phase rectifier so get 3 diodes. Also have two 50,000uf 100vcaps on pv output- a leftover from when I used classic pwm wastenot as relay control. A 20A 200v ssr inserts the heating element in circuit enabled when PV in rises about 7v over the mppv of the arrays.
I have not seen the Classic drop out of float (yet!) when the heater ssr enables. its only a 800w heater so the two atrings, 1400W, seem to easily cope
When experimenting with this setup I was concerned about the Classic and its possible reaction to seeing PV voltage pulled down as heater came online. I used the AUX1 float output on another ssr to disconnect the PV bank from the Classic. In normal mode the float high is used to keep the ssr closed and current flowing from PV strings to Classic. When float V reached this AUX1 goes low disabling the ssr and
disconnecting the two PV strings from the Classic. Shortly after as the PV voltage of the disconnected strings rises fairly quickly the ssr to the heater activates.
I gave up on the ssr disconnectig scheme and for simplicity just went with the one ssr inserting the heater. The Classic's mppt logic is smart enough to always select the higher voltage input from the two strings not connected to the water heater, as least so far when I have seen it scanning
This is why I am interested in a potential black box that could more efficiently use the available power. I'm simply now inserting an 800w parasitic load on the output of a couple of PV strings. The real situation is that when in float on a good day there must be over 2Kw available so what the heater does not use is wasted.
dgd
OK. Severe case of "Feature-Creep" here.
This was going to use an 08M2, the tiny little 8-pin device.
I'm thinking I might have to go up to the 14M2.
The chip as an ADC output. It's not very precise, but it's reasonably accurate.
It has 32 steps.
I wonder if I could make the differential amp a summing differential amp, and add my chosen offset (from the ADC output). This would let the PIC ramp the reference voltage up or down to monitor a 2V span from 0-2.048V up to 2.048-4.096V (theoretically), with the 2mV resolution.
Or, do I simply set the differential amp gain at 0.5, thus giving me 0-4.096V with 4mV resolution?
Do we feel the need to measure the temperature, and if so, do we need one sensor per cell? Per group?
When do we stop? Do I add three chunky FETs so we can try to automatically balance the string?
Do we try to crawl first, then see what else we can do?
Feature creep always an issue
It's one of the challenges to determine how many different scenarios to cover
Aside from 3? Or so lead acid battery chemistries .. Then there is LIXx ..... And that covers
A few more permutations
Ross - there is now a separate thread for battery ADC
Zone blue has started a couple new threads - which well help us better organize
.. I've already wrong posted today also !
I agree - crawling first is good
Just need to define "crawling" .....
IMHO
Keep temp sensing separate - it doesn't have the common mode isolation issues,
It's simple and already quite easy to do, and add x number sensors as users requires
Eg daisy chain 1wire DS18xx sensors along the cells cases
DGD
Thanks for detailed explanation - I can see the "limitations" in your current setup (no criticism intended) .. I'm impressed by your creativity ... And the approach to the "high voltage switching "
Challenges ... 150 VDC and big currents ... Not for the faint hearted (or inexperienced)
I see why/how the BBP. Could benefit your scenario
- once we have more intelligence available to us/BPP on opportunity energy .. Then we can
Chase the holy grail ... Use every drop available!!
Thinking aloud .... yes I know I have too much thinking time .... It's only a temporary aberration)
The BBP "vision" statement could include;
chasing the PV RE holy grail of "use every available watt of available energy at all times"
LiOn/LiPo etc requires voltage sensing up to 4.2Volt (the max allowable for these type of cells)
I sense ross has one foot already over the balcony rail :)
Ok guys, you are getting way ahead of a lot of us interested bystanders..fast, real fast....
so is it possible to start new BBP sub topics for a flow chart?
and (not sure of the right term here)
a schematic of all the different hardware that is being proposed
ie, rPi, BBL, Arduino, Cubie, etc et al.?
These in addition to the "Vison Statement"
Not sure about "flow chart"
But have a look at my thread "BPP sensing modules"
My attempt at some "structure" (at least for my thinking)
Hi Greg, yup, that was what got me thinking of a chart to keep all these bits organized for us old farts that think a bit slower :-[:'( than you bright sparks ;).
My mind started to implode :-\ :-\ after I had read all the overnight postings from you 3 leaders...
Not as young as I'd like to be !
Time diff gives the advantage .... You get to read them "all" when you awake I'm guessing
I'm working on some more for tonight - to fill out suggestions for HW options for each
Of the modules Under my proposed structure
Quote from: RossW on October 18, 2013, 08:28:35 PM
*ding* Blinding flash of light.... use the "top" of the previous cell! Ie, differential amps. Each one references its output on the negative of the cell under test!! 0-2V (nominal) out. I will still need a small bias or divider to be able to measure over 2.048V (really need to get to about 2.5V). Simple 4:3 divider with 1% resistors still leaves me around 3mV/bit resolution. If we're aiming for 10mV or better, we're comfortably in!
Just knocked it up, quickly. You will need a java-enabled browser to view this circuit simulation:
Even the forum software is breaking it!
http://support.rossw.net/batteryzero.cgi (http://support.rossw.net/batteryzero.cgi)
DoubleClick on the batteries to change their voltage and see the effect on the output.
I'm thinking that I'd use an 8-pin RESNET for each opamp to ensure they all track together, which means the output will be 1:1 for the inputs and I'll need a divider or level-shifter.
Got some tricks up my sleeve to optimise the ADC range, will see if I can pull it together some time in the next week or so. (Will be interstate for a few days from tomorrow)
OK, so I thought I'd knock up a schematic while I was having my coffee.
The picaxe 08M2 will (just) do what we want, but only bare-bones. Having two input-only pins cramps my style a lot.
So I looked at the price - and the Australian distributors want about $4.10 for the 08M2 and $4.70 for the 14M2. For the extra functionality I think I'll just skip the 08M2 and go straight to the slightly bigger one. For a 24-cell (48V) bank, that means 8 boards, so about $5 difference in component cost. "no brainer"!
So, I've dumped my original schematic and will go back to scratch. This chip has enough outputs to put useful indicators on it too.
Ross, did you delete the contents ? (link in #92)
Quote from: Westbranch on October 19, 2013, 07:01:46 PM
Ross, did you delete the contents ? (link in #92)
No, but it looks like the process of making it a tinyURL did. I'll try to fix, thanks for the heads-up.
Quote from: Westbranch on October 19, 2013, 07:01:46 PM
Ross, did you delete the contents ? (link in #92)
Sheesh. Tinyurl broke it. The forum software itself breaks the raw URL. Another URL shortening scheme didn't work. Tried to paste the code directly into the forum without the URL tag and it wouldn't copy-and-paste properly either. Just goes to show, when you want something done right, you have to do it yourself!
http://support.rossw.net/batteryzero.cgi (http://support.rossw.net/batteryzero.cgi) works here at least...
Ross,
Your link redirects to falstad.com circuit simulator site that seems to be overrun with ads for Asian women profiles...
dgd
Quote from: dgd on October 19, 2013, 08:19:05 PM
Ross,
Your link redirects to falstad.com circuit simulator site
Yes, it's the sim - the parameters (the "circuit itself") is on the rest of the (long) command line.
If you don't have a JRE, it won't spawn the applet though. (It should start Java in new window)
OK, here's the *START* of what I was talking about - but no circuit simulation.
Those of you familiar with opamps will see what I'm doing :)
Before you complain about the zener - if this has to work and allowing for one cell to be dead short (0V), we have less than 4V we can be sure of. And the device needs a minimum of 2.3V to work at all, and more likely at least 2.6V to be able to monitor a cell thats around 2.5V, it doesn't leave much for a regulator! Add to that, most regulators take at least 3mA (even a 78L05) which is more than we wanted to blow.
With 6V (nominal), we'll be dropping 0.9V across R1. Some of that will be from the circuit itself, the rest may be from the zener - if I need 1K, then that's 0.9mA quiescent. Probably acceptable. There's scope to improve on that by adjusting R1 in production once we know the actual current draw.
FAR FROM COMPLETE, just a discussion point.
(http://support.rossw.net/batterymonitor.gif)
Ross
Great progress - I think the current drain is acceptable - after all - any battery has a "self discharge" rate
Even when at rest/unloaded ... I'm guessing we are playing in that range
I realize you are still in design phase ... But thinking ahead to "implementation"
1) can we put a " disconnect" switch on each module - that would enable it to be switched off or /into a Very low power draw mode - that way if for whatever reason batteries were to be left in a situation
With little or no charging - the module could be put into a very low draw (zero?) mode
2) we need to think thru the fuse aspects - eg each sensor line to battery should have a fuse inline
I'm guessing for best practise - probably as close to terminal (which means no impact to your design)
3) isolation
- your idea of "potted" sounded interesting, I guess that would mean a set of flying leads emerging
From the pot (may be best idea to preserve isolation integrity in lieu of using a screw down connector block )
And a design question
Can the LED proposed be a tri/color or similar? I know you are thinking of using a series of flashes
For flashing status ( brilliant)
..... But if the LED could be "biased" (not sure the correct term) to also
Have red or green (eg a very visual way to show "all good" or "Houston we have a problem")
I think I see a spare output pin? Perhaps a second led in red? (Note we don't
Want to add to current draw if we can avoid)
My 2c worth
And at Xmas could we have a different flashing sequence ?
Only kidding!!
But it did make me think what a 48V. battery room might look like at night :)
The LED as drawn is purely indicative of function. It wouldn't be like that on the final design because it would (probably) draw more current than the series current limiting resistor could manage. For the minimal cost of it, I was thinking an open-collector transistor which could drive LEDs (plural) to full brightness off the entire "nominal" 6V supply.
One of the LEDs (probably only fitted to the last device in the chain) would be a higher-intensity IR LED for communications to the "outside world".
red/green LEDs probably. Not sure about running a RGB if we may only have a little under 3V supply (this thing may even have to run off around 3V, someone could have NiFe, NiCad or NiMH cells...
I was thinking of board layout and perhaps having a large through-hold plated hole, large enough for an M10 or thereabouts size bolt. One either end of the board, so it could be bolted directly to either + or - of a cell.
Disconnect might be a little tricky if you want to disconnect the sense lines too, or do you think that the miniscule load imposed by a 10K resistor is ok? (I could up the resistors to 100K, should still work fine)
I already plan to put the device to sleep after its done its thing. That will reduce its drain, but the zener is still going to be chewing up the lions share of it. I considered using an opamp as the voltage regulator, but the rail-to-rail devices I've got have a Vcc limit of 6V, so no good there.
Almost all the pins on the PIC can be used as inputs or outputs, so there are plenty of spare pins.
Direct connect to battery terminal could work - but it would then need an onboard fuse
For that connection - and then still needs a flying lead to the other batteries I assume ... We are dealing with gigantic arc welders here! any fault is going to turn the module into molten goop
(If anything left at all) ... Fuses are our savior
I looked at the current draw from just having the resistors and opamp load ... Perhaps I missed
Something, because I looked at it as being very high impedance ...
Either way I was thinking of the power draw from having the chips powered and of course Zener/voltage ref cct powered up - so I was thinking a spst toggle in the postive rail from highest
Battery eg prior to R1
I was thinking of disconnect on the supply rail only, and use fuses for complete manual disconnect from
Battery if required
Running of "3V " .... Yeah I guess your right eg fully discharged NiCd @1Vpc x3
Wow you are really designing for all cases
this raises a question - how are you planning to "program" the device for different battery
Chemistries? (Eg if your planning some basic on board fault /status indicators .... The voltage
Thresholds will need to change accordingly (DIP switch? Pin headers ?)
what's the high end limit?) I'm guessing the more
Likely future scenario will be increased use of LIXX cells (aka need to measure up to 4.2Vpc)
.. Ok ... I think I hear you say lets crawl first..
No mention of the Xmas season mode - very dissapointing :)
Quote from: Gregy on October 20, 2013, 09:58:47 AM
Direct connect to battery terminal could work - but it would then need an onboard fuse
For that connection - and then still needs a flying lead to the other batteries I assume ... We are dealing with gigantic arc welders here! any fault is going to turn the module into molten goop
(If anything left at all) ... Fuses are our savior
Fuses protect wires. (think about it!)
Quote
I looked at the current draw from just having the resistors and opamp load ... Perhaps I missed
Something, because I looked at it as being very high impedance ...
Either way I was thinking of the power draw from having the chips powered and of course Zener/voltage ref cct powered up - so I was thinking a spst toggle in the postive rail from highest
The devices should draw milliwatts. The regulator is almost certainly the most hungry element by far.
A switch or jumper is fine though.
Quote
Running of "3V " .... Yeah I guess your right eg fully discharged NiCd @1Vpc x3
Wow you are really designing for all cases
No guarantee we'll get all bases covered, but to knowingly design without considering them is foolish.
Quote
this raises a question - how are you planning to "program" the device for different battery
Chemistries? (Eg if your planning some basic on board fault /status indicators .... The voltage
Thresholds will need to change accordingly (DIP switch? Pin headers ?)
what's the high end limit?) I'm guessing the more
Likely future scenario will be increased use of LIXX cells (aka need to measure up to 4.2Vpc)
.. Ok ... I think I hear you say lets crawl first..
My intentions (initially at least) were:
* Let each board collect and report the voltages of its own directly-connected cells
* Let each board receive the status of every other board
* Have a threshold which applies across all boards, which each learns and passes on.
Initially, I was only intending to display an alarm on imbalance, not on high or low absolute values - that can be done by the monitoring/logging device.
To program - a simple RS232 input. (They'll already be communicating via serial, it's just a matter of levels)
Alternatively, the "first" board could have a 20-turn pot on it that is used to set the maximum delta. Lets say it read a proportional input as 8 bits. I can't imagine where more than 1V difference would be acceptable, so lets say the pot would trim from 0-1V in 256 steps - 4mV/step. To set, wind the difference down until lights start to pop up across your bank, then back off a bit.
Running of "3V " .... Yeah I guess your right eg fully discharged NiCd @1Vpc x3
Wow you are really designing for all cases
No guarantee we'll get all bases covered, but to knowingly design without considering them is foolish.
Perhaps dip switches, set by user, to cover the range/max V needed?
Quote from: Westbranch on October 20, 2013, 04:24:31 PM
Running of "3V " .... Yeah I guess your right eg fully discharged NiCd @1Vpc x3
Wow you are really designing for all cases
No guarantee we'll get all bases covered, but to knowingly design without considering them is foolish.
Perhaps dip switches, set by user, to cover the range/max V needed?
I don't like DIP switches. It'd take an 8-way DIP to get the same results as a trimpot, but far less intuitive to set up. A 20-turn pot would be basically "Turn fully counterclockwise. Now, turn 1 turn for every 50mV"
You want your threshold at 0.25V? Turn it 5 turns.
Remember those beautiful precision trimpots you used to be able to get? The calibrated ones with the black and brushed aluminum knobs. One of those would be sweeeeeeeet. Probably can't get them anymore 'cept on ancient equipment off <popular auction site>.
Quote from: atop8918 on January 15, 2014, 06:28:04 AM
Remember those beautiful precision trimpots you used to be able to get? The calibrated ones with the black and brushed aluminum knobs. One of those would be sweeeeeeeet. Probably can't get them anymore 'cept on ancient equipment off <popular auction site>.
I do, and I've got a few of them.
In the end, I opted for an entirely software solution. No trimpots to drift or take up board space, or add to the cost.
It is simply a command. Using a terminal, send S100 (for "Sensitivity, 100mV) or S50 (for 50mV) etc.
The figure is stored in flash memory so becomes "permanent". It's flexible. It won't drift or be bumped.
It's been near 2 years since this thread was active so has there been any further developments on the black box concept?
I realise I'm a late arrival here but it's just now that time has become available to allow me to become more involved. So it would be nice to hear about further developments, anybody?
Having reviewed everything posted here I have briefly listed what I would like to see, some of this I have already largely coded and it's now in transition from Arduino DUE to Cubie2.
- one second running data from Classic and displaying this in some sort of gauges web page or in a shifting leftwards multi data item graph.
This page should be configurable to use different gauge/graph types, like selecting a skin for the display. This already looks feasible with skins ranging from various text fonts and sizes, different available gauge types and many available moving graph types ( r charts, high charts etc)
- data summary page from 1 to 10 minutes per entry covering at least current days activity
My favourite here is the text table, about 10 minute resolution, web paged with up to 8 hours to 24 hours worth of entries.
A graph version too although higher resolution one minute makes a nice multipoint smooth-ish display
Ten minute or greater resolution looks nicer in bar and column graphs
- half day or day summary, again in text table format or graph covering 10 to 60 days.
The high stock charts seem the way to go here, very similar to the mymidnite graph screen
Although these are the reporting options I wanted to have and thought I would find useful, a couple of months usage and quite a few ideas from others (mostly email from people not using this forum) have brought up other ideas, The skinning of the live data display being the most popular followed by a display configuration option. This would use a config file to somehow create a user designed web page containing the data, live or historical, the user wants, in a possibe mix of gauge, graphs, text and even user images..
The mixed data page shown by ZB in the first post in this thread is such an example although this looks a fixed design
The issue I see with this is that its more a web page development issue than anything to do with RE/MN Classic application
With the Arduino DUE web server I have installed five of these servers with people/companies I did Classic installations or supplied the RE equipment. Discussing the web page design with each I ended up with about 8 different web page designs
(everybody has their own idea of what is the best design) so although its easy for the developer to make just one design available it becomes a take-it-or-leave it option for the end user, even just a minimal user driven layout change greatly pleases end users, providing multiple display configuation options involves hem in the web page design process.
So for the web side of the Black box a very user configurable page design would be my favoured way to go.
Another black box design concept is NOT a running or historical data reporting box (that information junkies really like) BUT an exception reporting system that operates as a quiet black box, monitoring various aspects of the RE system and ONLY generating some output when an exception condition occurs that wil need user intervention.
A minimal system example I have seen for this used an RPi that simply monitored battery voltage and if it exceeded upper or lower limits a cellphone text message was sent, a more complicated example had current sensors on each PV string and if
the current between them diverged past a % then again text and email was sent.
Dgd
It appears there is not really much ongoing interest in this project, lots of great ideas and discussion in many of the older postings but just nothing this last year or so.
Perhaps everyone is just too busy, not interested, or can't see any use in this project.
Time for me to give up too :-\
dgd
Hi David, I'm still interested to see where the Blackbox might go, but I know for me, I have many things going on right now that I need to work on.
My personal interest will still be with QuadlogSCADA and progressing the development of this, using the Cubieboard II as my test-bed for a low power consumption system.
I'll need to go back over what and where Blackbox was heading to have that clear in my mind again.
I like that there are a few clever and passionate folks over here, such as yourself, working hard into the early hours on bettering systems, and I would very much like to know that this would continue.
____
Paul
Ive still got energy for it, just swamped at work this year. With the AXS port now sitting in its box, support for Outback stuff is on the cards. Ive also been working on independent SOC, and thats meant a few changes to the core library in order for the current day to know something about yesterday. Also google code is being closed so ill have to move the SVN someplace else.
I would like to have made some major progress withe BB project but have run into some major issues with the Classic reliability. (next post)
Also I'm getting more enthusiastic about constructing my Oztules 6Kw sw inverter using big toroid transformers from deconstructed InspireSolar GTIs (rewinding two to make one) and Chinese PowerJack inverter electronics. I'm all the time thinking about getting a cpu into it, lots of sensors and making inverter data available via ethernet (but no modbus!)
This project came along at the right time as I was just about to purchase a Schneider SW 48volt 230v 50Hz inverter or an OB 3648.
I'm also seriously looking at designing/constructing a 120 to 160 amp PWM solar controller.
So perhaps later this year others may have more time to get back to the BB project
dgd
Agreed, time has just been completely consumed with other, critical/important tasks....
Perhaps by the time people can look at the BB project again, MN will have managed to resolve the Ethernet issues with the classic.
This one major issue has probably been the most significant in preventing realistic progress with this project (which should have progressed to completion a year or two ago).
I have not managed to get a classic SL yet, can't find anyone selling them. It will be interesting to see if it has a new design of CPU circuit board without the Ethernet hardware etc or if it's the same pcb just different firmware.
If it's firmware then has it been rewritten or just a hacked down version of the existing firmware, meaning that it's likely non-Ethernet related random resets, watchdogs resets, random clock instability, mngp randomly losing its ID (eg unused address X messages), stats/counters corruption on resets (like kWH total being annoying zeroed despite a simple solution of storing it EEPROM etc..), will continue to show up on the SL
I'm getting more of the opinion that just a total rewrite of the classic firmware is the only long term solution but then this could be impractical as the Classic may be on the downhill side of its product life cycle.
Dgd
Hi, I recently purchased a midnite classic and as I'm a linux guy and unwilling to send all my data to a 3rd party service the blackbox project is a good fit for me. It lets me monitor my classic from home or away. So thanks for building it guys.
btw, I'm a newcomer and I'm not sure what the ethernet problems are about. It's been working well enough for me for a couple months, though I do notice it seems kind of slow reading, and I had to fix an issue with the graph not generating sometimes due to php default script timeout of 30 secs. I'm even accessing it via ethernet over powerline to my solar shed.
I submitted a minor patch for the blackbox project installer several months back. As far as I know it was never reviewed or accepted. Then about a month ago I tried to submit another small patch but google code wouldn't let me because it has been discontinued.
I am hoping that the project maintainer (dgd?) can move it over to github, and at least accept some patches, even if no time for active development.
For some reason i didnt get notification of your patch, but then google seems to have decided to kill off its code hosting....
The raw data is stored in the classiclogs table, and there is a running agregate maintained in classicdaylogs. At the end of the day the last entry wins, and remains as a long term daily datapoint. I dont think theres any need to agregate further, the week, month and year agregates can all be derived from the daylogs in realtime. So relatively little code is needed to produce the summary graphs, built in the same user definable manner as the per day graphs.
For the graphing there are uncommitted changes that implement flot, which removes the overhead of the raster graphing, which as you found is rather cpu intensive on the small boards. If you like i can email you my trunk version.
I only used google code because of their SVN support, which as an old timer, im more familiar with. Maybe theres some adapter or something. Re git, you seem to have some experience with it. Maybe youd be interested in helping with the move.
So exporting to github appears to be super-easy. On the google-code page they have added a button "Export to Github" which apparently anyone can use, since I see it and I am not a project member.
Basically, one clicks that button, then "Start Export". An existing github account is needed, which is also very easy to setup.
I have a github account and am happy to do it if you wish. The only thing is that you are much better qualified to review and accept patches than I am, so it kinda makes sense to me that the first "official" blackbox repo on github would be yours. Anyone else with a github account (like myself) can then fork it, make changes, and then submit a pull request for each change. The pull requests contain a nice visual diff. I find it easier/smoother than using svn once you get used to it a little.
Vice-versa, if I export it, you could always fork my repo, and continue on.
Let me know how you would like to proceed.
Well the preminary migration is done.
https://github.com/zoneblue/theblackboxproject
When i can figure out how to commit using tortoise ill bring the code up to date.
ok, cool. I will fork and submit pull requests for a couple small patches I have.