Arduino based PWM diversion control system

[WillEert]

Got my copy of dgd's RS 232 modbus to classic system working but I am having an interesting problem with it that I do not have the knowledge to resolve. I am posting to that forum topic - RS232 modbus to Classic- as I think that might be more suitable. Thanks dgd for putting together such a nice system for people to use.

Will

[WillEert]

Having been successful in getting dgd's system working on the lower rj11 Classic port I now need to get the temperature corrected Battery Voltage setpoint. Using the code in the Uno via rs232 to Classic I was able to add a packet and get register 4244 into a second array. It took several attempts but I was able to make this work. Having done this I added a page to the 16X2 lcd and displayed this value - then added some math and calculated what will be the diversion set point for this control. This I also displayed on the new page. Realizing that I needed access to the pwm pins that the lcd shield covers up I changed the code and put in an I2C 20X4 lcd which displays the Classic mode, battery voltage, temp corrected battery voltage set point and diversion control voltage setpoint. Now I am pondering how to add the PID PWM control to this system. Ordered another book which is focused on programming arduinos -still lots to learn. I am hoping to use this control as a test bed for the controller that Paul is working on the code for. I think adding pages to a 4X20 lcd and buttons and lead/lag element code will be beyond me for a while.

Will

[paul alting]

I can see Will is making great strides with working with Arduino systems and attached peripheral components, well done Will, and it looks like I'll have another set of helping eyes.

Will, as far as putting the code for PID and PWM into the code you have is quite a step up in terms of the code logic required to do this.
The Arduino has built in functionality for analogWrite(x), which is PWM to a digital pin, but I have found this technique is not suitable for controlling a SSR.
I use a different method.

For me, I do need to remain as focused as I can on the code I am developing for you that does this.
Progress on the diversion code is getting there, the PID and PWM is in and operational for two load heating elements.

I am making the code such that it is able to be used by folks with Midnight Classics or Morninstar TriStar charge controllers, with separate Modbus templates for each.
The path I am taking with it allows for other solar charge controllers to be easily implemented and simply selected by a selection number at compile time.

Others might not yet know, but the code so far, which is designed to operate on an Arduino Mega, also has an integrated GUI system with communications on the main USB serial port via VT100 / VT220 terminal codes. This GUI allows for character placement at x , y screen location as well as colour to any standard ANSI terminal application.

In **nix based systems, such as Mac OSX or Linux variant, you simply use the terminal application and connect to the serial port and hey presto, a simple but worthwhile GUI on you computer screen.
With windows users, I have looked into this and it can be done using the built in 'command.exe' application or any one of many terminal emulators available that support VT100 codes.
Additionally, with this GUI, I am putting in place the ability, not only to view all data real time, but also to be able to alter setpoints and have other forms of control directly from your connected computer.

The code also has a full LCD menu system, allowing the same functionality as the VT100 GUI above, via a 20 x 4 character LCD and a number of push buttons.

My next step is to develop code to simulate a Classic charge controller so I may start testing.
For this project I had to swap out my Arduino Ether-Mega that was being used for my own running system for the Ether-Ten, a UNO with Ethernet size board.
This has now given me access to the Mega that I need for this project.
I have an older Arduino which I will setup as the Classic simulator over the next day or two.

The code for the diversion project continues to grow in size, but there is a lot of pre-processor code to help out with the particular user configuration at compile time.

I'm working toward having something available soon to show with code and GUI.
____
Paul

[WillEert]

Wow. The diversion system Paul is developing is a long way past what I imagined when we started chatting about making code for a PWM PID diversion system for my power supply. It sounds like a lot of work and effort -Thanks Paul. I like the idea of the versatility that the control that Paul is developing can be used by others, even with other brands of charge controllers people may have ( deluded though they may be ). I believe that for off-gridders a generic diversion system that they can obtain and will meet their needs is something the industry lacks. I know for my system my base daily load is about 5.5 -6 KWHr /day. With my old Apollo system that is what I made if my batteries were charged at the end of each day. Using the Classic , diverting excess energy via the Aux 1 Waste not hi function, into my HWT I now make about 12 -18 KWHr depending on the initial temp of the HWT - quite a bit more output from the power system. I use a hydronic heating system that extracts excess heat from the HWT and uses it for heating the house we are in the process of building. This works automatically turning heating on / off based on thermostat requirements and tank temperature. I have made 24 KWHr in a day while doing an equilization and heating as well. People pay for a power supply system and then the trick is to get as much out of it as possible to lower the cost per watt. The system Paul is developing looks like it will be able to do this on a generic basis. Great!

Will

[paul alting]

Thank you Will for your kind words.
Currently buzzing along on my little Mac mini, with around 6 (check: make that eight) virtual screens all running, including watching the last mountain stage of Tour de France, three screens for web browsing, a few others for skype, email and then Eclipse IDE, and another for the terminal GUI.

So, I have the basic code for a Midnight Solar simulator up and running in a small Arduino now.
I have this connected to the Ether-Mega which is operating the diversion project.
Yes, they are talking, which now allows me to further progress the code and do testing.
All is good
____
Paul

[WillEert]

Well I got the first iteration of my diversion controller running. My system is presently set up for AC diversion as I wanted to be able to use conventional AC thermostats on the tank. My next system may be DC diversion using an arduino based thermostat to control a suitably sized DC SSR. It took me a few days of struggling to get the PID control to read the values of the setpoint and battery voltage variables but by studying my books and trying various things finally a light bulb went on and I was successful. I put the Timer 1 library in quite quickly and set it up. I am quite pleased with the results. I have done some loop tuning in the past and it took a little bit but the tuning is not too bad I think. I am running the SSR at 10hz and the Magnum 4448 inverter is very happy at this speed. Thanks Paul. I had dedicated an inverter to diversion because of lights flickering / discomania at the Aux2 PWM speed but I think I will change my wiring around and see if I can my system working with all of the inverters operating in parallel again. The pwm always has output == 0 within -.3 or so of SP and there is no risk of draining the batteries. I have had a few sun cloud days and it seems to work well. Code posted. The board is an Arduino Uno R3. The 4x20 lcds are by DFRobot as is the RS232 shield. I made my cable by using a db9 - RJ11 adapter that acts as a breakout box. This was about $5 from ebay. Code attached.

Did you know that if you plug the 9V jack in and out of your UNO enough eventually the magic smoke will come out of the m7 diode and your UNO will stop being able to get power through the jack? I have learned this.

The control continues to develop. I have put in two tuning speeds in order to cope better with sun / cloud and when a large load starts or stops ( well pump, etc). I put in a second PID and logic to start it when the first is at 100% and then to turn off again when its output is zero. So far that logic looks like it is working OK. Now I have to rewire my HWT to simultaneous element operation and then get the second relay running the upper element. That will give the lead / lag operation that Paul suggested and that I like the idea of so much. It is apparent with the PWM that the tank temperature changes little if at all until the wattage the element gets is about 50% of rated KW. I picked up the thermostat I needed the other day so time permitting maybe next week I will be able to test lead / lag operation.

Bought another book - this one focused on actual C coding as it relates to the Arduino. I am a few chapters into it now and it is teaching me lots that has been a mystery until now.  I know the code that I have created is clumsy and likely does meet any number of good conventions. It does work however.

Will

[dgd]

Will,

Thats excellent progress 

The power connector problem seems to be well known on the Arduino newsgroups.
I always use the USB connector to provide 5volt power to the uno, Mega or DUE. This has several advantages.
It bypasses the on board voltage regulator which at best is not very efficient, ok for playing about with and occasional use of the Uno but as a long term power supply its not too good. The recommendations are a 7.5volt to 9volt DC power unit but if you have attached one of more expansion cards then it can take power consumption well over an Amp, this means the simple onboard regulator is probaby disposing of 2.5 to 4 volts at > 1A and producing several watts of heat.
My first Mega got very hot after an hour or so and it eventually died, completely dead the cpu is non responsive.
So if you use USB 5volt power = amost no excess heat at the Uno.

I also means to upload sketch that I just unplug the one metre long USB cable at the apple power wallwart and plug it into my laptop for the upoad.
I also use an apple 12/24volt cig lighter power adaptor, I have wired a twin USB socket version of this via 5amp breaker to my
DC power distribution panel. Works well, these adaptors seem to use a decent switch mode regulator so produce very little heat from my 24v battery system.

Although I designed my ex-pc-psu case with a teensy fan and the DUE with temperature probe to control it, it has never come on since I moved DUE to 5V usb power. After running a few weeks I can touch the DUE and its hardly warm, the warmest part is the Wiznet ethernet chip on the Enet card.

There is no such thing as code that works being clumsy, its always good code if it does what its supposed to. Be pleased with what you have achieved. Just remember to puts lots of comments in your code to remind yourself what the code is doing and how it does that. Comments are not for other people, they don't matter 
I know (hired) professional programmers who keep two copies of their code, one detailed with comments for their own use and one stripped of/devoid of comments that they distribute to others. Weird science eh?   

dgd

[paul alting]

Will,

Well done so far, good to see your progress and and achievements thus far.

I took a little look at the program and, well, over a cup of coffee I couldn't help myself with a bit of massaging of it.
I hope you don't mind, instead, hopefully it gives better insight.

I totally refactored the code to be indented to the normal 2 space and correct indentation for statements and for easy reading

The last two lines in the main loop() would be better moved to another location where they will do something while in the loop()
I am not able to make a lot of sense with the second last line. Having two equates on a single line is not usually done in C and is advised against such statements.

Looks like David has just made a post, so I better post this to read
Must be breakfast time, surely.

Will, I will have this code of mine ready today for testing, if you are keen to try when you are ready ?
I will prepare a write up for it.
____
Paul

[dgd]

Will and Paul,

I really like this code you have both posted and I will be setting up a water heating system at a friends place and this is just getting to what I need.
In the quest for a truly black box I wil be interested in not having an LCD display on the final production version, maybe at most an externally visible LED or two, one that perhaps flashes at a rate that reflects to PWM cycle and another for power.

It seems the Classic has no issues with using both available serial ports to talk to Arduinos. I have tried DUE for web server and an UNO on other port just showing data on LCD. This UNO I''d like to get to manage the DC water heating.

dgd

[paul alting]

David,

Yes, in regards to the LCD, I agree, and have been thinking on this in the past weeks while writing this code of mine.
In the past, I have used LCD, such as 20 x 4 style units and they have performed well in those applications.

On my HydroSolar system, I do have an LED that does light in beat to the PWM, and it provides a nice quick indication for me of the battery state.
When it is not flashing, I right away know that no dumping of energy is taking place.
Then, while it is beating to the PWM signal, I get a fair idea of the amount of dumping taking place, with an almost on beat telling me that a lot of energy is being diverted to the dump load.

In the code I have, I have a pre-processor define that can be set at compile time to set either the LCD code or the VT100 code.
Presently, I just use the VT100 code as I think that is a nice way to display data as compared to a LCD.
Plus, I don't have a LCD2004 here at the moment anyhow.

The VT100 code is simply the Arduino sending out data through the normal USB serial port connected to your computer.
Your computer runs a terminal application that understands VT100 escape sequences.

I'll get back to doing some finishing off things to the code and then make a post with it all.
One point, the code is set for an Arduino Mega, not a UNO, but if you really would like or need to have it for a UNO, I think we an do that.
Using a UNO, with only one hardware serial port and wanting to use the VT100 GUI is going to need to use the SoftwareSerail library.
I haven't used this library as my thinking is that if a project needs multiple communications ports, then use a bigger board that has the ports.

It will take some hours yet, and have fresh coffee to make a start
____
Paul

[paul alting]

I was on a roll and simply wasn't able to resist in distilling your code even further Will.
It is about half the size now and a lot of unused variables have been removed.
I also re-wrote a few of the variable statements for efficiency and readability.
I removed a whole switch statement also, by integrating its code into the first switch statement.
Hope you won't be mad at me, but now, for me, it has better flow and is readable, I hope you might agree.

With comments, my take on it is that comments are there for you as the coder writer, but also very much to help others understand your code as well.

Right, back to what I should be working on, distractions distractions .. 
____
Paul

[WillEert]

Wow - lots of response to this iteration of the as I call it, UDC - Uno Diversion Controller.

Dgd : Thanks for the positive comments. I am always happy to learn when and where I can. I will search out your system for powering up the Arduinos and implement it as eventually the control will be mounted in a box nicely instead of sitting on a chair bread boarded together. Not cooking my boards has to be a good thing.
       : LCD's - for tuning purposes I put in 2 lcds with many parameters so that I can watch the control work and try to optimize it. I would not think others would likely want or need all this info.
       : I had a version of this control operating with 2 tuning speeds however in the development process it went away. There is an example in the PID library that shows what needs to be done to get two tuning speeds. The control works better with this in place. My tuning for the second speed is Kp =600, Ki=112.5,Kd=.6. Not completely sure of these tunings as when I see it go to fast tuning it settles out so quickly I have a hard time seeing what it did.

Paul: Thank You for reworking the sketch and posting the results. I am not angry but happy. I have learned quite a bit by analyzing what you changed and will continue to ponder the changes. I have not had time to look at the second interation yet. I tried to compile the first one but it would not compile for me but it is not the compiling but the changes that matter. I struggle to understand a lot of the compiler error messages in the IDE.  I have found a free debugger for C that is supposed to be very good - http://visualmicro.codeplex.com/. Have not had the time to even go to that site yet however as time permits will try it out. When I used to do build process simulation models for industry I would build myself a little control panel in order to make it easy to change variables. I can see that by moving setpoint variables into the initialization part of the sketch it makes it easy to make changes without having to find the variables each time in the sketch. Yay.
         : I am eager to put together the hardware for the diversion control you have coded up and try it out. I see that you changed its name. When you are ready so am I. If you let me know what it uses for an lcd then I will order another one if I need to as I want to continue to operate the udc as we develop this more sophisticated pwm pid diversion controller. Thanks

Just for fun here is the 2 pid 2 speed sketch I am working on now. So far it is running well with the understanding that I do not have the lag element physically hooked up to the relay yet. I am biasing the  diversion voltage setpoint of the lag controller ahead of the lead controller to get the lead / lag response to be as it seems it should be.

Is coding an addiction... I am starting to think of so many things that can be done. I am thinking about putting the narcoleptic library in the sketch in order to sleep the controller while the Classic is resting.....


Will

[paul alting]

Hi Will,

I'm still up and hard at it, and haven't yet replied to your emails yet.
I have the first release for you to take a look at.

The code is fairly intense in relation to the code we have so far been working with, so, strap on a safety harness and parachute as well

The archive file attached has a number of files you will need.
It includes the libraries for an updated TimerOne, DDModbusMaster and BasicTerm.

These libraries are in a folder called private_libraries.
Take these library files and place them into your private Arduino library location.

Next is the actual project files, and these comprise the main file and header as well the included files.
From the archive, copy the folder called 'includes' and placed this within your project folder itself.

Finally, copy the two main files, called main.cpp and main.h to your actual project folder.
Rename the main.cpp to main.ino for your Arduino IDE to work with it.

Now, I don't recall exactly, but you need to tell the Arduino IDE to import the three libraries, so it knows about them.
I'm sure you will have already done this before, so I'm hopeful you will be ok there.

Open up the the file called 'modbus_maps.h
You will see in here that all the actual modbus mappings and functions are done.
In this file there is three templates for three different modbus devices, a simulator, the Midnite Classic and the MorningStar TriStar devices.

At line 32 of this file you set the device you are using.
It is currently set to 10 to select the Midnite Classic template, like this;

Code Select Expand

#define CHARGE_CONTROLLER_TYPE 10

In this file, all the communication parameters are set, such as baud and slave ID.
The slave ID is the Modbus slave ID and is set to 1.
The baud is set to 19200, which I believe is correct for the Classic.

You should be able to compile and run this code in your Arduino Mega.
Full control is set for controlling 2 load elements in a lead lag method.

With lead / lag control, the lead output is always the output that is modulated by PID and PWM.
The lag output will only ever be on or off.

If the lead output gets to greater than around 95% of PID output, then the lag output is switched on.
The lag output remains on while the lead output continues to modulate.
When the lag initially comes on, the lead will need to wind down very fast to maintain the load.
The lag output will stay on while the lead output has its PID output greater than 5%.
Once the lead output PID falls below this value, the lag output turns off.

Now, there many need to be some off delay function on these controls yet, but let's try first with this.

The whole show can be controlled via a terminal that emulates a DEC VT100 or greater.
When you upload the program to the Mega and then operate the terminal application you will see an image like below on your screen.

The settings for the terminal screen are baud of 115200 and 8N1 for data, parity and stop bits.
I selected the faster speed to help things along.
The onboard LED of the Mega will flash for every poll of the charge controller, which is presently set to every 250mSec.

To connect the Mega to your Classic, connect up the RS-232 or EIA-485 to the Mega's serial 1 port.
Serial 0 (zero) is connected to your computer and serial 1 is the next available and is configured to talk to your Classic or TriStar.

Once you have your VT100 terminal application running and can see the screen as below, you are up and running.
At this point, you can enter various commands, which I will list briefly here.
Each command has a 10 second timeout.
Pressing '+' is used for increment of setpoints, and '-' is for decrement of setpoints.
You need to select a setpoint first before you can adjust it.
Press 's' for the diversion controller offset setpoint.
Press 'p' to select the PID P term.
Press 'i' to select the PID I term.
Press 'd' to select the PID D term, though this won't do a lot and is advised to leave at zero.

The PID is a full integer PID loop controller that I programmed up some years back based on fragments of other code.
So, the three terms are all integer.
I have found for best results, the P term to be between 5 and 20.
For the I term, keep it low, between 1 and 10, works well.

On the right side, you will see the two outputs and I have included a basic bar graph to indicate the level of each.
Of course, the lag output will only ever show on or off, but the lead bar graph will grow in proportion to the output value.

You can swap the lead and lag outputs by pressing the 'l' key, that is lower case 'L'.

You will see the Modbus poll counts increment on your screen with every poll of the charge controller.
I had a small problem that I only just found today with the Modbus library and have tried to contact the author about it.
It was polling at full speed, ignoring the configured poll rate.
So, for the past week with my testing, it has been polling and absolutely hammering my poor little Arduino UNO simulator.
But, importantly, not one error was recorded. Not in over a few hundred thousand polls at breakneck speed.
Tonight, I found the the problem and corrected the library to now function as intended, a nice loitering speed of 250mSec.

There is more to write, but my brain is fading at this hour.

Is coding an addiction, you ask, well, what can I say, I think like David, we are mad, totally mad, but enjoy every moment of it.
I still like to get my hands dirty in the garden or go for a nice long ride on my motorbike and tent to get away from it all.

Well, its' 01h:30 here, I'll attach the archive and image and hit the sack
_____
Paul

Edit: I forgot to add, the two output PWM pins are pin 11 and pin 12 on the Mega.
These are connected to Timer One output compare capture registers.

Edit2: More. If you are using a Mac or GNU/Linux, then you can simply use the terminal application as part of the operating system, it's all built in.
Go to the terminal app and list the connected devices, by;

Code Select Expand

ls /dev
Look for the connected USB device and then connect to it this way;

Code Select Expand

screen /dev/cu.usbmodem621 115200Where cu.usbmodem621 is my connected device, yours will be a different ID.
Works beautifully
For Windows, I'll have to research what's available and best, but I did look at those horrid site where you can download windows software, looking for a terminal application.
Maybe some of you folks will know what is a good terminal application to use.

[paul alting]

It's morning and there is snow outside, first time in 15 years 

For Windows folks, maybe this terminal application is good, Tera Term ?
Or is the name Tera not meaning Earthy, but more along the lines of Terrible or Terrifying ?
http://tera-term.software.informer.com/
A link on that page takes me to this site for more information http://external.informer.com/o/ttssh2.sourceforge.jp/index.html.en/AA001290

I'm not sure how or if you can trust those sites or how else to get application software for Microsoft Windows systems.
It brings back bad memories from many many years ago when I did use it.

Ah, looking and searching further I find a better site that talks about Tera Term and gives more background information.
From this page you should be able to get the software and install it to have a quite nice terminal that will work for both serial and SSH modes.
http://logmett.com/index.php?/products/teraterm.html

When you first connect to the Arduino Mega that is running the code, it will have already started after you press the reset or upload.
The screen will need to be re-freshed and this can be done by pressing the 'c' key or by again, resetting the Arduino while you are connected via TeraTerm.

There is more work I need to do on the code yet, but this first release should be usable and give results.

Right, it must be coffee time for sure and to see if more snow is coming
______
Paul

[paul alting]

I should mention again that the code I develop, I do in Eclipse IDE, not the Arduino IDE.
And in the Eclipse IDE, I choose to use indentation using tabs, not spaces.
Tabs work fast and allow me to quickly format my code into a very neat appearance.
Spaces need a lot of tapping and I would soon ware out my pinky and space bar, or cursor keys if I did have the tab reinterpreted as spaces.

What does this mean you ask. Well, the code which appears nice and tidy in my development environment will not look so great in another IDE that does not display the tabs correctly.
The Arduino IDE is a real good example of this, and it pains me to see folks struggling with this IDE, even for a short time.

Like Will has mentioned a few posts back, he was unable to successfully compile the code I had massaged.
The Arduino IDE does not highlight potential errors as you type.
It also lacks a wealth of features that really make programming so much easier and to the point of fun.
And yet, they promote the Arduino IDE as a friendly way to get started into programming micro-controllers.
There are many things about the Arduino IDE that are good for beginners, but the things that make it a struggle to use will, for the foreseeable future can only hamper its usability and friendliness.

While I say this, I also don't mean for everyone to jump on the Eclipse wagon, not unless you are really keen or see yourself doing more and more complex coding.
If you do, then I can suggest Eclipse is just pure joy to use.

I guess with my use of it, I see most of my errors while typing the code up.
Also, when I compile the code, it highlights exactly in the code by underlining the erroneous section with red.
This saves me having to wade long lists of text and trying to decipher the compiler messages.
Any error after compilation, I can simply hover my rodent over it and a popup will appear explaining the error of my ways.

Using Eclipse also allows me to write code for Arduino, and any number of other languages all at the same time.
This allows me to quickly flip form one project to another, whether that be between any number of different Arduino projects or javascript or HTML or PHP or Python.

So, returning back to my reason for this post, I had Eclipse print out the code to PDF files.
I have attached the two most important files here, being the main.cpp which handles all the main logic of the project.
The second file is the code that handles all the Modbus part of the project and is an include file, or a sub file to the main file mentioned.

Looking at these files will be far easier on the eyes, unless you have a suitable IDE such as Eclipse to view the code.
_____
Paul