Status of the MNS COMBOX Project

[Wizbandit]

I know many are waiting but seems like little issues keep cropping up...
So here is the present status:

I got lazy in the initial code and the way I was getting data from the devices.  See, the CANBUS has hundreds of packets of data being "streamed" every minute.  Me not wanting to sort through all this data for the few bits of information I needed, I decided to just "ask" the devices for what I wanted.  Well, seems the MNGP2 does not like "others" asking for data that is available on "the stream", it gets confused as it thinks it should be the only one "asking".  Now I had to rewrite the whole code base, building "CANBUS Engines" to filter out and grab only what data I wanted for each device.  The new engines run in the background and update data periodically and it is stored in the database.  I also optimized code to shut down and release memory & resources for devices that are not actively running on the system.

The new system has been running for 4 or 5 weeks now with almost no glitches, it's time for me to go back over and double check the new code.  The Classic module is a big mess so I started from scratch on it and while I'm at it, adding code to get it as a device on the new MNS CANBUS as another supported MPPT device.  This is a daunting task but I'm making progress.  I have built almost a complete MPPT Charge Controller in software that the MNGP2 can't tell from a real one.  All I need to do is read the MODBUS data from the Classic and "attach" it to the proper CANBUS registers and bing!  Classic on the CANBUS.

Everyone have a nice Holiday and I will try and post updates more often...
JIm

[Raster]

Is it possible to support CanBus batteries (such as EG4-LL V2 48v) in a similar fashion, or are they already supported by Rosie/Barcelona?

R.

[aaapilot]

Is it possible to support CanBus batteries (such as EG4-LL V2 48v) in a similar fashion, or are they already supported by Rosie/Barcelona?

R.

I don't believe any battery systems are supported as yet in a closed-loop connection to the CANBUS.  But, it's a great question.  I have the new Pytes V5 batteries and would like to have closed loop capability as well.  Perhaps the re-written software will allow the data from these streams to be "attached" to the CANBUS registers just like the Classic data?

[boB]

"I don't believe any battery systems are supported as yet in a closed-loop connection to the CANBUS"

Actually, some of this has been done but I don't remember how many different batteries ?  Maybe I never knew ?  But there is some work going on here. I'm just not privy to knowing which one(s)

No EG4 closed loop yet as far as I know.

It's going forward though and more and more BMS's should be added

boB

[aaapilot]

Actually, some of this has been done but I don't remember how many different batteries ?  Maybe I never knew ?  But there is some work going on here. I'm just not privy to knowing which one(s)

Great news!  Looking forward to this update.

Dave

[Wizbandit]

Yes, it's been a while since I posted an update, I work part-time helping out in the tech support department, things are really busy with the new products... and old...

Okay, here is where we are at:

HARDWARE MODIFICATIONS - Adding a 9V battery, charging circuit and detect for power gone for safe shutdown to avoid LINUX file system corruptions. I updated my Rosie's yesterday and forgot it shuts off the power and BAM, crashed the COMBOX LINUX file system.  I had to make a new Micro SD card and set it back up.  I suspect only a small percent of our COMBOX owners will be able to do this and we would need to mail out a replacement Micro SD card every time.  I have a prototype on my bench with a bread board working on it now. Maybe an option for user supplied 9V non-rechargeable battery, I'm using a slightly expensive 9V lithium with BMS built-in.  Shipping devices with lithium battery might be a problem to some locations.

SOFTWARE UPDATES - For the first units this will need to be done by USB thumb drive, copy the update file, plug it into the COMBOX, load the config page and select USB UPDATE. The software code for this still needs to be done. Next step is to add code to update from the Midnite Server and leave the USB option for sites without Internet.

STILL to DO:
1) Add option to set the CLOCK without an active Internet connection.
2) Finish adding the Classic to the CANBUS so MNGP2 can control it.
3) Rework code modules for DIY3024, DIY3548, DIY5048.
4) Add module to access the NEW MNS AIO 10K Inverter.
5) Get user feedback for options.
6) Update and finish the manual...

[ClassicCrazy]

Yes, it's been a while since I posted an update, I work part-time helping out in the tech support department, things are really busy with the new products... and old...

Okay, here is where we are at:

HARDWARE MODIFICATIONS - Adding a 9V battery, charging circuit and detect for power gone for safe shutdown to avoid LINUX file system corruptions. I updated my Rosie's yesterday and forgot it shuts off the power and BAM, crashed the COMBOX LINUX file system.  I had to make a new Micro SD card and set it back up.  I suspect only a small percent of our COMBOX owners will be able to do this and we would need to mail out a replacement Micro SD card every time.  I have a prototype on my bench with a bread board working on it now. Maybe an option for user supplied 9V non-rechargeable battery, I'm using a slightly expensive 9V lithium with BMS built-in.  Shipping devices with lithium battery might be a problem to some locations.

SOFTWARE UPDATES - For the first units this will need to be done by USB thumb drive, copy the update file, plug it into the COMBOX, load the config page and select USB UPDATE. The software code for this still needs to be done. Next step is to add code to update from the Midnite Server and leave the USB option for sites without Internet.

STILL to DO:
1) Add option to set the CLOCK without an active Internet connection.
2) Finish adding the Classic to the CANBUS so MNGP2 can control it.
3) Rework code modules for DIY3024, DIY3548, DIY5048.
4) Add module to access the NEW MNS AIO 10K Inverter.
5) Get user feedback for options.
6) Update and finish the manual...

Can you build string of MQTT or option for that into the output so we can integrate into our existing monitoring systems ?
Will the Canbus protocol ever be published like the Classic modbus has been ?
Thanks for your efforts - will look forward to trying out whatever comes out of it.
Larry

[Wizbandit]

I did install a MQTT Broker or "server" as it is now called on my development COMBOX, it is running but I have not tested it.  It claims to be a "low bandwidth" service but that does not mean it does not need a lot of processor to operate.  As it just serves to distribute "published data" to "subscribers" with no data being stored on the actual server I would need to publish to it first.  With over 400 different data points I don't want to publish every one as I suspect that would load down the system to a halt.

The only way I can see to accomplish this is to have a configuration page for each device, Rosie Inverter, Barcelona, Hawkes bay, Classic, KID, DIY3024, DIY3548, DIY5048, 10K MNAIO-ONE.  Each page would allow the user to turn "on" MQTT and select using "check boxes" the data they would like published to the broker, select too many and the system crawls to a halt.

I still need to work on "safe shutdown" hardware, with the increase in support calls I have had little time to work on this issue.  Once it is done I could start on building the configuration pages and testing.  You would need to have some way to have your external monitor systems access the broker as I won't have time to work on outside devices.

The MNS CANBUS is complicated and not real easy to understand.  Where as you read a 16 bit MODBUS register and get for example "Battery Voltage" our CANBUS is 64 bits and each of these packets can hold from 1 to 8 bytes of data.  For example register 0x0A0 is the "battery register", it has 2 bytes for voltage, 2 bytes for current and 4 bytes for power (watts) stored in "big-endian" whereas MODBUS uses "little-endian".  I know this won't mean much to many of you but the bytes need to be combined to get the one data value.

MODBUS data is requested per 16 bit register whereas CANBUS data is "streamed" and you need to monitor for the data you want which requires multiple tasks running concurrent in the background.  That being said there are configuration registers that must be requested and they return up to 64 bits of data that needs to be unpacked and combined.

It took me over a year to become really good at this...FYI

[ClassicCrazy]

I did install a MQTT Broker or "server" as it is now called on my development COMBOX, it is running but I have not tested it.  It claims to be a "low bandwidth" service but that does not mean it does not need a lot of processor to operate.  As it just serves to distribute "published data" to "subscribers" with no data being stored on the actual server I would need to publish to it first.  With over 400 different data points I don't want to publish every one as I suspect that would load down the system to a halt.

The only way I can see to accomplish this is to have a configuration page for each device, Rosie Inverter, Barcelona, Hawkes bay, Classic, KID, DIY3024, DIY3548, DIY5048, 10K MNAIO-ONE.  Each page would allow the user to turn "on" MQTT and select using "check boxes" the data they would like published to the broker, select too many and the system crawls to a halt.

I still need to work on "safe shutdown" hardware, with the increase in support calls I have had little time to work on this issue.  Once it is done I could start on building the configuration pages and testing.  You would need to have some way to have your external monitor systems access the broker as I won't have time to work on outside devices.

The MNS CANBUS is complicated and not real easy to understand.  Where as you read a 16 bit MODBUS register and get for example "Battery Voltage" our CANBUS is 64 bits and each of these packets can hold from 1 to 8 bytes of data.  For example register 0x0A0 is the "battery register", it has 2 bytes for voltage, 2 bytes for current and 4 bytes for power (watts) stored in "big-endian" whereas MODBUS uses "little-endian".  I know this won't mean much to many of you but the bytes need to be combined to get the one data value.

MODBUS data is requested per 16 bit register whereas CANBUS data is "streamed" and you need to monitor for the data you want which requires multiple tasks running concurrent in the background.  That being said there are configuration registers that must be requested and they return up to 64 bits of data that needs to be unpacked and combined.

It took me over a year to become really good at this...FYI

I think there is a difference between publishing the data and having a broker ?
I am no expert.  I use Grahams Classic MQTT software and pretty sure it does not publish ever data point available - just the ones most of us would want to use - voltage, current, soc, whizbang, state of charge, etc.
I have the broker running on my raspberry pi - so don't need one running in a combox.
Anyway thanks for including mqtt - it will make life a lot easier for anyone wanting to monitor data.
As far as Canbus - I gave up on trying to figure it out for the Hawkes Bay because my old brain isn't up to the task now.
Larry

[Graham]

I agree with Larry, having a broker in the combo box doesn't help us DIYers. We need to integrate data from various sources for display in grafana/influxdb or load/diversion control in node-red or homeassistant...
Also if the broker is in the "cloud", it allows us to access the data from anywhere on the internet...
I have a couple of projects in Github that convert serial data from Jakiper lithium batteries to MQTT and modbus data from the classic to MQTT, now we'll need to have a Canbus to MQTT converter...
It would be great if the Combo box dealt with all the complicated Canbus comms and included a simple MQTT publisher to forward the relevant data to a broker.
An MQTT publisher feature should not be a load on the CPU (I have light bulbs that publish MQTT).

Graham