Encryption OFF

[dgd]

I am able to redirect the data streams that emanate from my Classic destined for the MyMidnite server.
The data appears to be encrypted and I need to decrypt it or have the encryption process in the Classic disabled.

Is there an modbus register than can be set to do this?

If not, then what information do I need to do the decryption on one of my network computers? and can a future firmware update include such a register?

Also any info on the data format of the data that streams to the MyMidnite server every 10 minutes would be useful.

dgd

[atop8918]

Hi, DGD,
The data stream is just modbus encrypted using 3DES. The keys are unique for each Classic. There is no method for turning the encryption off at the moment as the information is already available decrypted through the standard MODBUS interface. If you are already redirecting the data stream you might be better off using your redirection to use MODBUS read your own set of registers and then send that data where ever you want it.

Thanks,
-A

[dgd]

Andrew,
I am already accessing raw modbus data but with the Classic limitation of one tcp connection I realised the MyMidnite was using a second  tcp connection not available for other processes.
Now if I could make use of this encrypted data I could have two reporting programs accessing just one Classic - one live data dashboard display and the other a reporting grapher.

dgd

[atop8918]

That's true. Unfortunately it would be very involved to redesign the mymidnite interface to make for an on/off encryption stream. It sounds easy but there is a whole bunch of handshaking that goes on to establish the connection. I don't think we'll be able to offer an encryption-free stream in the short-term.

[zoneblue]

Or if you had the DES key you could decrpyt it...  Still for most datalogging purposes the fixed 15 minute interval is likely to be too long, yeah? 

[atop8918]

Well that's true but you'd also have to implement all the handshaking in order for the Classic to talk to your application as well. We were intending to build a PHP package that you could install on your own web server that would do all that for you but we're still swamped at the moment and that is down the bottom of the list.

[zoneblue]

It seems to me that the "combox"/blackbox route solves all of these problems. 

However given that most folk need plug and play, its either gona have to be a midnite undertaking, or a very comprehensive community one.

In terms of the community effort, the various hacks and patchs that we are all doing is certainly not fast tracking development on this, and no faster than the midnite timeline.

What to do?

[dgd]

Well that's true but you'd also have to implement all the handshaking in order for the Classic to talk to your application as well. We were intending to build a PHP package that you could install on your own web server that would do all that for you but we're still swamped at the moment and that is down the bottom of the list.

Handshaking processes are quite normal in data communications applications.
If some documentation were provided on the existing handshaking requirements that currently exist between the Classic and Mymidnite server along with the appropriate decryption key information then  I have no doubt we (users) could do something useful.
Please take 10 minutes to provide this info.
Thanks.

dgd

[dgd]

It seems to me that the "combox"/blackbox route solves all of these problems. 

However given that most folk need plug and play, its either gona have to be a midnite undertaking, or a very comprehensive community one.

Sadly neither of these options are viable.
Midnites often mentioned paucity of resources means glacial progress on their non front line undertakings.
The community efforts are laudable but with very very few with the expertise to complete a commercial grade combox for MN products (Classic and Kid) then this approach too is sporadic at best.
Nevertheless its probably all we will ever have.

dgd

[Halfcrazy]

Well you know what they say. You cant please everyone.

We have tried to make a very nice charge controller that works well and has functions that everyone wants. Of course that's not possible so we do as much as possible and also keep it affordable. Its true there are things it does not do that some would really like but then there are others who like it as is. So at the end of the day we have to look at a few factors to decide what to do. We have to decide where our money is best spent if you will. One goal is obviously to make enough money to stay afloat. With that in mind a company really needs to balance Feature creep and new products. It is very easy to spend hundreds of thousands of dollars on feature creep for 1% of the users.

So with that in mind the Classic is not done by any means but we are focusing the majority of the engineering resources towards new products. We will continue to make changes and add features to the classic though and we truly value every ones input.

So if we have severely let down any of our customers I truly apologize but we are balancing our resources as we see best.

I will chat with Andrew about the handshake data and see if I can get that info.

On a side note we are going to put a call out on the KID for some one to program a "communications adapter". Our goal is to have it speak modbus like the classic and maybe in that box there is enough resources to fulfill a lot of requests. As soon as Mario gets some serious serial data flowing We will start a thread on this and see who can help move that project forward

Ryan

[dgd]

Thanks for your reply Ryan.
My goal here is to encourage some more open information on the technical side of the Classic.
Although the Classic's comms limitations are well known (ie one tcp connection) this does not detract from the fact that the Classic is a well featured and quality solar controller. I have not seen anyone disappointed with it yet.
So please don't think I am somehow disappointed, quite the opposite in fact.
Now that I have an opportunity to play (for 2 days) with an eTracer controller and just started with a Swiss made Studer innotec controller I can even more appreciate how far out in front of competition Midnite is.
Just the addition of the WBjr and soc info leaves everyone else way behind.

On that side note, have you (MN) drawn up a general specification for the proposed MNcommbox?
I'm assuming you need the processor to maintain memory registers compatible with the modbus spcecs or MN version thereof. Serial input/output via USBs to Classics and KID? and perhaps data archiving.
Maybe an inbuilt web server? 
dgd

[zoneblue]

Why isnt supporting say blackbox an option? You seem to have energy to try to decrpyt the mymidnite stream, and hack about your system generally.

The reality is that the classic really has a vast array of communications options, more than any other device. True none of them are "fisnished", and it could be said that resources are being spread thinly over too many options!

It seems to me that the "combox"/blackbox route solves all of these problems. 

However given that most folk need plug and play, its either gona have to be a midnite undertaking, or a very comprehensive community one.

Sadly neither of these options are viable.
Midnites often mentioned paucity of resources means glacial progress on their non front line undertakings.
The community efforts are laudable but with very very few with the expertise to complete a commercial grade combox for MN products (Classic and Kid) then this approach too is sporadic at best.
Nevertheless its probably all we will ever have.

dgd

[zoneblue]

On a side note we are going to put a call out on the KID for some one to program a "communications adapter". Our goal is to have it speak modbus like the classic and maybe in that box there is enough resources to fulfill a lot of requests. As soon as Mario gets some serious serial data flowing We will start a thread on this and see who can help move that project forward
Ryan

It might make sense to have such a thing be able to talk to both Kid and Classic? 

[dgd]

Why isnt supporting say blackbox an option? You seem to have energy to try to decrpyt the mymidnite stream, and hack about your system generally.

The reality is that the classic really has a vast array of communications options, more than any other device. True none of them are "fisnished", and it could be said that resources are being spread thinly over too many options!

Ok, I'm not being critical of anyone's efforts here BUT
the blackbox concept seems to centre on a physically small limited resource single board computer. Albeit one really useful for a process control application but more often touted as an educational tool.
Having bought into the rPi and its Linux, I quickly discovered its unreliability and performance issues.
I gave up and its in the junk box.  Great educational tool (its primary purpose?) but doing anything real world is painful.
Maybe the others are better.
So my thoughts moved to the blackbox being a software project. Just run it on a proper computer with decent memory, storage, cpu power etc and the project reduced to just reading modbus via serial port or TCP.
No critisism of Ross but I didn't really want a command line modbus interface, I wanted to compile modbus functions into my higher level reporting progam, written in C,C++  I know I could have simply used a 'system' call within my code to Ross's newmodbus but the performance issues with exiting to a shell command and paramter passing is something I don't want.
My coding  developed into my simple  C modbus library for Classic. Other C libs for modbus just were way too convoluted and complicated for me.

Now to the Classic, no matter how smart the code is I write and does what I want, the overwhelming and crippling limitation of the Classic is its ONE and ONLY TCP connection. I cannot have a permanent PC connection via TCP to gather and live report or archive data as I can then not connect remotely with say the Local App to get a snapshot of how things are going.

This is why I am interested in this MyMidnite connection.
I don't use MyMidnite, its too limited for me to find useful (no negative reflection on developers intended)
BUT it does have a second TCP connection reserved for it in the Classic.
I would like this TCP connection made available for other purposes.
Ideally the Classic firmware has its TCP connection limit changed for the current setting of ONE to a setting of TWO.
Failing that then change the Classic firmware to allow the 'web access' system to be defined by the user - an MNGP function or Local App function for Lites.  Or the server name setup in a series of modbus registers so that those of us who prefer that method of changing the server name can do so.
Failing that then the required information on the Encryption key and the handshaking the Classic is expecting.  Its a dogs bollix of a job but it is possible to redirect the data to the MyMidnite server to a local network server.

Really the proper solution is the first option, increasing the TCP connection limit to TWO as we already know the Classic can deal with two connections.

dgd

[zoneblue]

Ok, I'm not being critical of anyone's efforts here BUT
the blackbox concept seems to centre on a physically small limited resource single board computer. Albeit one really useful for a process control application but more often touted as an educational tool.
Having bought into the rPi and its Linux, I quickly discovered its unreliability and performance issues.
I gave up and its in the junk box.  Great educational tool (its primary purpose?) but doing anything real world is painful.
Maybe the others are better.

Actually ive been quite impressed with Cubiebaord. But theres nothing at all stopping you running blackbox on any linux box, indeed ive also run it on my atom dev server, which draws 20W. It ran little better and drew 15x more current! What is it exactly you are trying to achieve that needs 2Ghz cpu and 8G ram? Theres some nifty new atoms coming out that may be a good comprise.

So my thoughts moved to the blackbox being a software project. Just run it on a proper computer with decent memory, storage, cpu power etc and the project reduced to just reading modbus via serial port or TCP.
No critisism of Ross but I didn't really want a command line modbus interface, I wanted to compile modbus functions into my higher level reporting progam, written in C,C++

I use newmodbus to talk to the classic via modbus, its way way faster than anything else out there, and rock solid. C is the appropriate language for that for sure. But i wouldnt code a web app like blackbox in in C, not in a million years. The work ive done on newmodbus, gets around the stack timeout issue, gives faster and more stable timing and now handles classic reboots gracefully. It transfers the entire register set to a text file, one address per line. All the web app has to do is read the file. But did i know C before i started that side project, no i didnt. Sounds like you know it though, and i couldve used your help.

I know I could have simply used a 'system' call within my code to Ross's newmodbus but the performance issues with exiting to a shell command and paramter passing is something I don't want. My coding  developed into my simple  C modbus library for Classic. Other C libs for modbus just were way too convoluted and complicated for me.

Well isnt that us all hacking away and not benefiting from the synergy we could acheive by  cooperating? To realise a community supported combox will require our mutual best efforts employed. Im not saying my way is the way, far from it, i merely opened up some code as a discussion point.

Now to the Classic, no matter how smart the code is I write and does what I want, the overwhelming and crippling limitation of the Classic is its ONE and ONLY TCP connection. I cannot have a permanent PC connection via TCP to gather and live report or archive data as I can then not connect remotely with say the Local App to get a snapshot of how things are going.

Thats the whole darn point of the combox, to allow multiple conections! The webserver in the box allows mulitple simulatnous connections from anywhere, and talks to the classic using its one free modbus link.

This is why I am interested in this MyMidnite connection.
I don't use MyMidnite, its too limited for me to find useful (no negative reflection on developers intended) BUT it does have a second TCP connection reserved for it in the Classic.
I would like this TCP connection made available for other purposes.
Ideally the Classic firmware has its TCP connection limit changed for the current setting of ONE to a setting of TWO. Failing that then change the Classic firmware to allow the 'web access' system to be defined by the user - an MNGP function or Local App function for Lites.  Or the server name setup in a series of modbus registers so that those of us who prefer that method of changing the server name can do so. Failing that then the required information on the Encryption key and the handshaking the Classic is expecting.  Its a dogs bollix of a job but it is possible to redirect the data to the MyMidnite server to a local network server.

Arent these the 1% type asks that Ryan is talking about? Id possibly go along with: if mymidnite is disabled, to allow a second modbus connection, if its practical to code. Itd make my life easier at the moment yes, but its not a deal breaker honestly. My present plan is pretty simple, to have a button on blackbox UI, that says enable/disable the classic connection. Using the local app is then a matter of clicking disable, doing your business then renabling it. Its not ideal but will work. As i said the other day, it would also be possible to build the local app config functions into the blackbox, and be free of Air for good.