The blackbox project

[RossW]

The latest trunk code has a solar insolation module built it. It calculates insolation from first principles, using a variety of algorithyms, the most promising of which are from pv-lib here:
http://pvpmc.org .Those are "clear sky" models.  Ross's insolation measuring approach is better.

Tried working it out but nature has other ideas. (Part of my application, incidently, was to quantify the effectiveness or otherwise of my 60-evacuated-tube solar water heater). Knowing what was actually there is far more relevant than what some hypothetical calculation says might be, if I were at 30,000' !!

I also have been reading about reference panels on the same plane as the array, and ordered a small glass covered mC panel from dx.com last week to test. From what i read about reference panels its the Isc that is of interest, so shunt, feeds adc is probably all you need.

Yes. (If you also want to use it for meterological work - calculating evaporation rates, growing potential, etc, etc, then you do need a proper, cosine-corrected, horizontally-oriented sensor)

Lastly, what if we had 12+ adc channels. Then we could monitor each battery cell's voltage. Now THAT would be cool. Pick up imbalances before they get out of hand.

Greg and I were talking about that in PM last night. I've a base design I plan to develop shortly, its been sitting collecting dust for 12 months but I really need to do something about it!

Ross. I havent hooked up the webcontrol yet, but i noticed in the manual, it says you can also add additional adcs using 1 wire, of which there are several ports on the board. Will that work?

A qualified yes. I am using it to monitor my 6 arrays plus array voltage (3 original ADC channels + 4 on an external ADC). Maxim however discontinued the DS2450, so it's not an ideal thing to build into new systems. Against that, I've found someone who has used a PIC to "emulate" the DS2450, using its own ADC and simulating a 1-wire device.

R.

[dgd]

OK. A few years ago, I installed the pyranometer at my own site so I could measure the actual power available at any given time. ONE of my purposes for this was to calculate the actual efficiency of my entire system. By measuring the power from the arrays vs what was actually available tells me how well I'm collecting power. By measuring how much my charge controllers delivered into my batteries tells me their efficiency, etc.

HISTORICAL INFORMATION IS USELESS here because it can't predict what atmospheric conditions we have (cloud, smoke, dust etc).

I also calculate every minute, where the sun should be and from that what the radiation should be. By comparing that to the pyro output, I am detecting (quite accurately) small amounts of "lost radiation" due to high cloud, haze, smoke etc.

Additionally, by knowing the radiation present, I am able to calculate a "predicted power" from my arrays, knowing where the sun is (by calculation) and how much sun there really is (from the pyro) and calculating the error angle (absolute off-axis vector elevation and azimuth) from each array to the actual sun position.

Between all these, I've a pretty good idea of just how much power I *COULD* be getting from each array (and in total).

i can see you have put in some serious effort to evaluate the solar potential. The only difference with my location in suburban west Auckland is that the amospheric conditions seem much less vairiable, negligible smoke and dust and minimal cloud - in fact I have recorded conditions over the last three years and have 85% clear sunny, minimal or no cloud conditions, in that time only four occasions where dense cloud cover and rain lasted longer than 3 days. The only disadvantage to this is that it takes an hour to get seriously burned by UVB.
This means I can depend on more stable predictions based on historical data...

Now, here's the kicker (the bit I haven't done yet).

As explained elsewhere by others, the classic is only drawing as much power from the arrays as it needs to supply the batteries and instantaneous load. If I "could" be making 3KW but the batteries only need 400W (because they're in float) and the load is only taking 800W (because that's all that's running now), I have a "potential" of (3000 - 800 - 400) = 1800 watts of "potential power" that I could be using for something else.

Yes I am well aware of this 'problem' as I see power waste all too often.
Just as an aside I recently decided to remake the mounting frames for my panels and after seeing the bank was fully charged I disconnected most of my panels while rebuilding frames at a different location (less shade problems). I kept 6 by 140watt panels in one string to a Classic250 connected. I was surprised to see that over 4 days this much reduced PV input still had the bank into float by midday and my plans to reduce load by turning off the beer fridge (and consuming contents!) was not necessary. My normal AC loads were much less than I expected and on reflection the process of replacing stuff with its lowest power equivalent (led lighting everywhere, 230Kwhr/year fridge/freezer, 229Kwhr/year LG 55 led TV, energy efficient washing machine etc.) had clearly worked!

Ideally, I should "switch on" about 1800 watts worth of opportunity loads.

It also needs to be "smart enough" to drop opportunity loads when we don't have enough headroom to maintain them all AND keep the battery and load supplied.

I do that for hot water. Now my rather simplistic HW controller just switches in half my PV array when bank goes to float and PV voltage rises sufficiently so that removing half PV array does not drop float. I currently use a eeepc that does nothing more complex than measure the PV voltage which rises when Classic demands less power to keep bank charging.
This simple method probably accounts for a large proportion of the possible 'excess' power but a more intelligent and accurate measuring method, black box based?, should allow greater use of the potential excess.

It would be nice if you and Gregy (and others) could keep your discussion here rather than by email as I, for one, would appreciate learning more about how to measure and get more efficient use of excess power.

dgd

(thanks TomW for moving this thread to a separate topic, I would have liked to see a separate water heating topic as after battery charging it seemed, to me anyway, that there was a lot of interest in using excess power to heat water BUT then again maybe not
as others have said here they want to use excess power for other purposes)

[zoneblue]

Ross, PVlib has a whole set of functions, which are well tested in the field, for taking the planometric insolation, typically from a meterological TMY file, but your plano is perfect,  and takes you through a whole series of adjustments. AOI, Reflection and refraction off the glass, airmass color spectrum, cell temperature, iv curve etc, and arrives at a dc mmpt output voltage and current. Its quite cool. Its coded in matlab, which isnt overly onerous to port.

The alternative is to actually use a 'reference panel' of the same type and characteristics on the same plane, and actually measure its output.  Both are worth pursuing imo.

[Gregy]

DGD
Over my above posts and the below, I will have replicated all of the PM exchange between Ross and I
As I was new to the forum - and had a few possibly whacky thoughts - I wanted to bounce of someone
Before potentially embarrassing Myself ...but I see you are all a great  bunch - so herewith all crazy
Thoughts will just appear regardless
... Now don't complain over size of my posts !

Zoneblue
I'm incredibly impressed by your coding effort and level of detail - I wish I could offer help on coding,
But found after one year at start my career that sw coding was not for me - these days I'm
High level only (a simple script is my limit)
I'm thinking through how I can contribute given various factors and location .....

To sensors
(Background - I've been using Dallas 1wire for 15+ years, and tried and used various others on home
Automation and weather collection system)

Temperature
** hard wired
DS18S20 / DS18B20 / DS1822
Hard to beat the Dallas 1 wire (provided that your HW has 1wire hub support - more later)
there are a few different models, thing to be careful is that some are parasitic (DS18S20) others not.
1wire sensors are exactly that, 1wire for data, and a ground (usually a second wire)
However some 1wire sensors  require separate power, as the bus "parasitic" power is insufficent, this
Also becomes an issue as both wire lengths and number sensors increase (then it becomes effectively
2wire plus gnd)
the CAI will only support non parasitic (DS18B20/DS1822)  max 8 - which slows down the CAI polling.

Analog and digital IO

1) DS2450 : analog input
was reported to be unreliable and susceptible to noise, volume consumption dropped
And as you correctly point out, maxim dropped it.
Agree we should steer clear of relying on this sensor.

2) CAI : I have not used it personally, it's (current model) 3analog inputs and 8 digital IO limits
Its potential as your system grows, and indeed in my system is way too small to begin with.
It has ethernet connectivity (good) and also has inbuilt web server (good) ... But of course must be
Connected to a hub/switch with Cat5 cable ... So if you want to meaure "remote" info,
It must be collocated (not good to locate the ADC inputs at any distance for the signal being sensed)
Because it's already an Ethernet device, working with it is quite easy AND it can operate standalone, and has some on board logic (PLC) like capability.
Summary : works well for local sensing or where your LAN goes, but it's (current) analog input (3) is
Too restrictive (note it's rumored there may be a newer version coming with greater IO)

3) Dallas 1wire digital IO
Various options and too many to discuss here, can be useful if 1wire bus already available

4) PICs / Arduinos / RPi (with shields)
Lots of different options available - these work well for low cost distributed nodes and for
Taking a lot of the low level sensing and calculations away from main Automation server
these are fast taking over - because of wide support, myriad of HW options, high density
Analog and digital IO, low cost, low PWR options, wireless support, ... Benefits are many.
As distributed sensing and IO nodes, IMHO - it's hard to beat where arduino is at.
Note options will differ widely of course, if you are experienced and competent in PIC, then why learn
Another etc etc
For me - I think arduino is best choice IMHO IMHO

5) arduino
Examples D= digital IO, A=analog Input. (Note also that some digital outputs have inbuilt PWM also)
UNO  14 D, 6  A. $30
Mega  54D, 16A. $48
Note the prices are for "official" versions, there are a bunch of 100% compatible Chinese versions
On ebay and elsewhere for starting at $18 for mega and $12 Uno
yes $18 for a device that has on board 54D 16 analog ADC capability
Breakout boards and sensor support is endless
AC Current via Non invasive CT, DC current with Hall effect (yes measure each of your DC loads up to 30A),,  gas sensors, high voltage isolation, current shunt sensor (a whole diff topic)

Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!
(However see my following post on this specific topic - a summary of the Greg and Ross offline discussion)

Now to BBP (I'm still trying to be on thread topic)
Linux based - already has support for 1wire library, I2C (talking to arduino), SPI and many others
if it's RPi (I recall that rpi was planned as one of the supported BBP HW platforms)
there is no on board ADC - but can be easily added with shields and/or "talk to an Arduino"

for "local sensing" it may be hard to avoid the simplicity of just adding a shield to Rpi (there are
Many choices to add multi channel ADC, expand digital IO, 1wire bus) - indeed I've seen one
That will do all this on one shield

For remote and distributed sensing, IMHO hard to beat low cost low PWR arduino uno (@$12)
There are even arduino nodes with inbuilt wireless avail for as low as $20

DGD - you asked for the detail ....

Zoneblue - separate question: if RPi is a planned HW platform for BBP, what are the options/plans for memory storeage support ?
I forsee that as sensors grow, storing all the data may outgrow on board memory cards
(Given what I've seen - I'm sure you have this already thought through .... Just curious your plan)

Should we have a different sub thread for "BBP sensors/IO" ?

[Gregy]

Zoneblue

One of my specific requirements is to be able to get access to BBP data into my separate home automation server
So I'm interested in how BPP would be accessed ( to read data ) from a LAN connected
PC

Noted your busy - so whenever you get a chance to respond - look fwd to hear further

[RossW]

Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!
(However see my following post on this specific topic - a summary of the Greg and Ross offline discussion)

I haven't done any more real thinking on this since we emailed last night Greg, but I did have one idea that I'll try to make some time to play with shortly.

Plan: a tiny, almost trivial, board using a PIC or PICAXE. 8-pin job. Has three analog inputs.
One device across THREE cells, can measure cell 1, 2 and 3, while being powered off nominally 6V.
Cell 1 directly into input 1 (about 2V).
Cell 2 through a 2:1 divider (about 2V)
Cell 3 through a 2:1 divider (about 3V)

This device reads the cell voltages and sends via serial "up the chain".

Next device - identical to the first.
Its "serial input" will now be flapping about between around -6 and -2V with respect to its own ground.
Simple resistive divider from second boards Vcc to input, and input to previous board, we should now have a sensible-enough input to read the serial.
It receives "Cell 1: xV", "Cell 2: xV", "Cell 3: xV", "End" via serial. As it receives each, it "passes it up the line".
When it gets "End", it adds its own three cells picking up from the last. (adds cells 4, 5, 6).

This ripples up the chain. Each little board does 6V, so 2 boards for 12V, 4 for 24V, 8 for 48V.
Each one has its own "local reference" so the problem of differential creep across 24 cells is reduced.
The last board will output its data straight to something to actually log/use. Perhaps by IR LED.

As I was typing, additional ideas sprang to mind.
Feed back the last board to the first. Now, every board knows what every OTHER cell is. It can compare its own cells voltage to the "across the entire pack" average and will know if a cell is out of balance - regardless of the SoC of the whole pack.
Add a LED. Now a board can indicate a problem cell in a stand-alone configuration. Walk out and immediately see. If board 2 is flashing its led two blinks, pause, two blinks, pause, it's its second cell that's crook.
Sure, monitoring, logging and alarm can be added, but something to show you immediately that there is a problem and which cell it is, a good thing!

I reckon I can make 'em on a sliver of a PCB that will be perhaps 3/4" x 1/2"

Will try to find time to lay some boards in the next week or two.
Anyone want to pre-order some?   

[Gregy]

Ross
I have a more detailed post under preparation (that covers various considerations - and the
Multi CH ADC challenges to zoneblue question)

However a few questions and observations
- what is stability/accuracy of the pic voltage reference?
- what is the value of Vref?
- what's the ADC resolution?
All the above will impact measuring resolution and accuracy.
Agree  that by reducing the number cells, this minimizes the differential creep
- PIC. (And arduno) ADC requires the input impedance to be around 10k
So a 2:1 divider will put a 20k load across each cell (for 2V cell that's  0.1mA "permanent"load )
I think this is manageable .... Probably within the self discharge/leakage range ... And if batteries "always"
Connected to charger .. Not an issue ) as some batteries will not have the divider, there is a
Slight imbalance in load/discharge between cells, but again it's very low
- I'm trying to get my mind around what happens to resolution with a voltage divider
With a 2:1 divider it halves resolution?
thus
10bit ADC using 5Vref gives 4.9mV resolution, for each differential measurement (2 measurements)  this reduces to 10mV (rounding to make it easier) .. And the the impact of voltage divider?  Does it again halve?
And hence is it effectively 20mV (I'm reaching for paper and pencil! ohms law will be my friend)

Q: what resolution is needed to be able to Compare cells for "problems" ?

we may need to be able to measure at 20mV resolution (or better) to be able
To meaningfully measure differential cell voltages (for 2V cells) - if resolution or accuracy to low
Then the results may not be useful ... And could be misleading?  (But at least would prompt a
SG test and a DMM cell measurement!!)

Edit
original post edited - removed reference to mfr referenced volt differential - as I think it was SG differential
That was being reffered to

[RossW]

I was planning to use a 3.3V (or close to) reference, because in a worst-case (single cell gone short circuit) I can only rely on around 4V to power the device.

Using 10-bit ADC would give nominally a 3.3mV/bit resolution. Of course, it isn't that good, so lets say <10mV/bit.
With oversampling we can crib some of that back.

With a 2:1 divider we're now back to around 20mV/bit.

Ways to overcome it. Easiest is to add a quad opamp, like MCP6004. Can level-shift rather than using a divider and back to 10mV/bit. If we're clever, level-shift AND add some gain. If we said 1.5 to 2.5V translates to 0-3.3V into the ADC, we have now 1V/2^10 bits, or near enough 1mV/bit over the reduced measuring range.

That also addresses the 20K load, but adds some cost, potentially some "calibration" required, and of course the device itself will be drawing some modest power (perhaps a milliamp?). The voltage reference/regulator will likely be the hog.

[Gregy]

Agree that using 3.3Vref is better - helps improve resolution

The figures are for "each measurement" ref to module 0V ?
Hence when doing the subtraction to get differential (eg individual  cell)
The effective resolution halves again?

I still think it's more than OK .... I've been doing some further research on the "allowable"
Differential voltages in 2V cells (not yet ready to put fwd a decision on this..)
But I think if we can be 20mv effective for a 2V cell (after all the halving et al) we have a useful
Measurement (if.nothing else at least use it as a trigger alert for "do a SG test" or further investigation required! )
.... But I really do want to find a way for us to capture this into a device for "automated alerting and monitoring" (visually checking a red led or similar is good ... When you regularly visit the battery bank,
But there will be requests no doubt for a more remote monitoring capability ... Me thinking aloud)

[RossW]

.... But I really do want to find a way for us to capture this into a device for "automated alerting and monitoring" (visually checking a red led or similar is good ... When you regularly visit the battery bank,
But there will be requests no doubt for a more remote monitoring capability ... Me thinking aloud)

Oh, the visual display was just an "added bonus". My intention was always for it to be externally monitored/logged.
The "end of the line" would operate an LED, which could be optically detected and fed into a conventional input on "something". Full isolation. Constantly updated.

[Gregy]

What's the max allowable supply rail for the PIC?
Remember an EQ charge can raise per cell Volts to around 2.57V (depends on battery mfr recommendations)
Still within the 3.3vref range for ADC - ok

But you could see supply rail of 3x2.57=7.71VDC

[Gregy]

Ross - I can't say I completely follow the design logic of the "serial up the chain" - you are way  more creative Than me!
And certainly you have good PIC experience ... So I'm mute on that aspect of design

But the multiple nodes - with flexibility for different cell voltage/string composition sounds good
And one node for every three 2V. Cells should give resolution and accuracy and balances the various
Issues we have noted, also makes self powered viable (need to check the average battery drain
When operating)

Break out your breadboard, soldering iron and PIC programmer
Look fwd to hear the results

Thread  moderator - I really think we need to break the threads up - as we are mixing up a bunch
Different issues and I don't want us to loose focus on BBP also
Perhaps a separate thread? Or place for "BBP sensors and input"
(Just a suggestion)

[Gregy]

Zoneblue ... As promised - some things to consider for battery cell ADC

Measuring battery array cell voltages

The challenges
- each cell (except for bottom cell) has a negative terminal that is NOT at ground potential
When things are referenced/connected to ground - it's termed "common mode"
- measuring individual cells in a string requires differential or other techniques (common mode
Technique becomes a challenge as the number cells increases)
- cell terminal voltages need to be "isolated" from ground in measurement sensor
- sensing equipment should put minimal load onto the individual cells (we don't want to create
The very imbalance we are seeking to measure!) ... And we don't want to drain the batteries unecesarily
- need sufficient resolution and  accuracy to be able to get useful data on cell imbalance
- typical ADC in low cost devices (arduino, PIC, CAI etc) can only measure 5V, for higher voltages
This requires use of a voltage divider
- voltage divider puts a load (albeit small) across the cell "permanently" (unless elaborate switching
Is used to disconnect when not measuring)
- to get high accuracy, each voltage divider needs to be accurately measured, or each ADC channel
Individually calibrated (digitally in sw) Against its respective voltage divider
- various schemes using relays are possible, but this method has challenges, not to mention lots
Of relay clicks and mechanical reliability

Common mode "subtractive method"
Measure each cell referenced to ground, and then subtract the next lower cell, to get the differential
Voltage of each cell (what I call the sequential subtractive method)
Works well for smaller number of cells, but as the number cells increases, the number ADC channels
Increases also
Eg for a 48V=24x2V  requires a 24ch ADC .... In a single device almost Impossible to find
Easy you say ... Let's add a second device .... Now this is where common mode isolation bites you
Eg Arduino mega has a max 16ch ADC (works great for all but 48V using 2V cells)
When you add a second board/module - it MUST be electrically isolated from the first.
Isolation can be achieved via optical, wireless or electrical techniques

one method is to use a number of isolated modules and measures a few cells with each module
Ross has proposed a neat way of using IR led from each sensing module to isolate it. (The optical isolation technique)

Now here's design considerations when using the above multi Ch ADC method
- Each cell needs an individual voltage divider referenced to ground (aka common mode) to
Be able to use a 5V (or lower) ADC .. So each cell has a current draw, and there are rules for stability on the ADC inputs that require a certain input impedance, typically 10k ohm)
hence value of resistors in volt divider will change (as you go up the string)
- ADC with 10bit resolution allows 1024 steps between 0v and Vref, if using 5v reference, this gives
Resolution of 4.9mV  (note I said resolution ... Not absolute measurement accuracy - which requires
Each ADC and input to be separately calibrated against high accuracy voltmeter)
Note accuracy is impacted by a lot of other factors (inc the Vref source)
- however the resolution halves when using the subtractive method (there are two ADC measurements
To determine each cells individual voltage)
So it can deliver 10mV resolution for each differential ( 5Vref) measurement
(See also the above various posts between Ross and Greg)

Dedicated measuring ICs
Various chip mfrs have solutions custom designed for the multi cell measurement
Environment ... But finding any commercially avail built up boards has yielded
Me no acceptable results to date
I've researched a few and one of the more promising is
maxim max14920/1 (12/16ch)
However to date no "commercial" solutions have been built.
Note there is advertised by maxim an "evaluation kit" ("max14921evkit")
That is fully built (supports 16ch) - but no pricing avail (could be a sticker
Shocker I suspect)
It has on board USB fully isolated (can stack 2 or more and avoid the common mode issue) and has win s/w driver and neat application for read and display cell voltages (perfect for win people,like me

Summary : for a BBP "battery string cell voltage measurement"  at present the modular PIC/arduino type solution appears the best
(Eg per Ross proposal and the above various posts)

However it's achieved - I remain convinced its a useful additional input into BPP
And can be used for monitoring, optimizing and early fault/maintenance indications
Of cells in the battery string
... yet another set of sensors for BPP to store (thus need to think thru how much
And frequency of historical data to store!)

[zoneblue]

Zoneblue quote "what if we had 12 ADC channels for battery measurement" ... See above!

Would another approach be to use some kind of multiplexer? Gees you know how it is, so i have 12 cells, the next wit comes along has 24 cells... A multiplexor into a single adc might scale. Battery cells obviously dont need a lightning fast sample rate.  Just thinking out loud.

One of my specific requirements is to be able to get access to BBP data into my separate home automation server So I'm interested in how BPP would be accessed ( to read data ) from a LAN connected PC

Thats the easy part. I envisaged a simple http API that can exchange data with other devices, in particular hosted servers that act as go betweens / backup / replication etc.

Zoneblue - separate question: if RPi is a planned HW platform for BBP, what are the options/plans for memory storeage support ?

Well, obviously with linux a whole rainbow of options is open to you. You can replicate data to a NAS already on your network, push it to any number of online storage options, dropbox, s3, whatever. Im sure ross would help with some cheap hosting for those that want to go that route.

Now of course we need to bear in mind,  at every step, our old friend power consumption. None of us wants to use a bunch of amps to measure how many amps were using.  I have a bunch of other computers running part of the day for various purposes, but i only want that one device running 24/7, so its got to be sub 4W total.  The cubie draws 1.25 so that leaves about 2.75W for the switch and the analog stuff.

A 32 GB SD card will store a fair bit of data. Lets see, the cubie's mysql database here is currently at 80KB for 75 days. At that rate, itll take... 1095 years to fill it. But for me that isnt really the issue, the issue is my low level of  trust in sd cards. Most of the newer gen arm boards have sata ports and the eg samsung 840 series SSDs idle in the 50mW range. Does that give you a clue as to where im headed with mine?

[RossW]

All good discussion.

I looked at some capacitive multi-pole switches for another application. These move both sides of a cap to the "under test" circuit, then quickly switch both sides back to "local ground" to measure without the common-mode errors. It can work with several hundred volts differential, and it's remarkably accurate. But with it comes a price. And it would require one per cell.

The pic+opamps solution is currently looking like a good start.
Only 10 bits is a bit of a pain, but there are things we could do to increase that. Oversampling, switching a small bias and resampling, etc. All trickery, but all valid. At the end of the day, how much precision do we really need?

Accuracy can be provided by calibration, but precision needs to be built in.

In my most recent considerations, a closed loop where every block knows the average voltage of each cell in the entire bank, makes for a very good "something needs a human to look at me" alarm.

I will be interstate much of next week and have a power of stuff to get through this weekend before I go, so unlikely I'll get anything done on it in the immediate future, but I've put it in the "think about it and get back to me" part of my brain. It should start popping answers at me soon