Bug reporting.....

[RossW]

Just informationally the Classic connects to the mymidnite server using DNS resolution so if you want to use your own MX entry on a local DNS server, you can certainly redirect your Classic that way.

Unless you're sending the information by mail, MX records are irrelevant.
A records, sure. But not MX

[dgd]

Just informationally the Classic connects to the mymidnite server using DNS resolution so if you want to use your own MX entry on a local DNS server, you can certainly redirect your Classic that way. The problem is that all traffic is encrypted using session keys so you would have to implement the decryption engine locally. We don't have plans to release our encryption algorithm at the moment -- and I realize there will inevitably be the security through obscurity comment, but we are using an industry-standard NIST approved technique -- so redirection might not gain you anything for the time being.

dgd -- the graph should update automatically when you move the navigation window -- which OS/browser/browser revision are you using so I can fix that for you?

W7 firefox latest updates of both.

Andrew and boB..
So Classic is assembling and sending an encrypted data packet to the mymidnite server. No modbus between them and the server info is buried in the firmware.
So what we need is a firmware update that defaults to mymidnite server OR an alternate user defined server address and if an alternate is specified then encrytion can be either on or off, if on then pgp with public key definitions  etc.. or the classic sends packet to a LAN encryption server, gets encryped packet back and transmits..
A data structure definition is needed too for the data packet..   you have this already..
Thanks,  asap please... don't you just love open software architecture.. 

dgd

[RossW]

So what we need is a firmware update that defaults to mymidnite server OR an alternate user defined server address and if an alternate is specified then encrytion can be either on or off, if on then pgp with public key definitions  etc.. or the classic sends packet to a LAN encryption server, gets encryped packet back and transmits..

If you control the destination server (if it's not midnite), then the necessity for it to be encrypted is somewhat lessened, because at least YOU have the choice. I accept that there is a lot of (processing) overhead encrypting - but perhaps even the use of (simple) SSL would be sufficient protection without undue overheads on the classic?

Hopefully TomW will rise soon (I see the sun came up where he is a little over an hour ago, so he should be about soon!) - and can p'raps send you a rPi executable to try?

RossW

[atop8918]

This is why we have an open MODBUS format which it seems some folks are already developing their own logging software. 

There will be an open API at some point to retrieve data off of mymidnite, and there's the MODBUS standard for accessing the Classic directly. SSL is too heavy for our processor with the rest of the processing going on internally.

I stand corrected on the MX records-- Ross is correct, it's been a long couple of years for me...

[TomW]

Just as an aside.

I have been pulling data from 2X classics via a Pi and crond job with Ross's little application every 5 minutes for days. This data is 100% there locally and when pushing to Ross's server it is 100% when my hillbilly can and string internet works. Which is "usually unless I need it"  .

Frankly, I backed off Mymidnite because it will not let me register my Wind Classic and the data is so sporadic on the Solar Classic I don't see any useful info in anything like real time. Mostly waiting for it to get sussed out rather than complain it doesn't work.

As Ross said, I have an ARM binary I can pass along if you want to try local logging via ethernet rather than USB. I found the USB method on the Pi seemed to swamp the communications via USB to storage devices quite regularly.

This little binary grabs the data lickity split and it is returned before the key click fades. It uses no external libraries and is wholly Ross's work.

Let me know if you are interested.

Tom

[Halfcrazy]

At some point I want to have a discussion with everyone but I think we could all benefit from a section here for all the open source stuff. I can see a ton of possibility's here with the PI and other stuff.

Ryan

[atop8918]

Hi, Tom,
I'm sorry that I haven't been able to free up your wind Classic. Please let me know the device ID and I'll see if I can free it up for you.

I'm also sorry if the data is too sporadic to be useful, this seems to be a common problem which is unfortunate as this is exactly what we are hoping to solve with the service.

[TomW]

Hi, Tom,
I'm sorry that I haven't been able to free up your wind Classic. Please let me know the device ID and I'll see if I can free it up for you.

I'm also sorry if the data is too sporadic to be useful, this seems to be a common problem which is unfortunate as this is exactly what we are hoping to solve with the service.

Got it now. That Local App device ID bug.

Patiently awaiting progress. You did open it as an unfinished project so no worries here.

My only feature request would be a way to name them or use the names they already have here. Solar, Wind and eventually the "Political Hot Air" unit which I expect to be a great power source once you sort out the algorithm!

Thanks.

Tom

[atop8918]

They should be grabbing their names now as programmed from the local app or MNGP.

-A

[dgd]

This is why we have an open MODBUS format which it seems some folks are already developing their own logging software. 

There will be an open API at some point to retrieve data off of mymidnite, and there's the MODBUS standard for accessing the Classic directly. SSL is too heavy for our processor with the rest of the processing going on internally.

I stand corrected on the MX records-- Ross is correct, it's been a long couple of years for me...

Having resigned to write some code to poke data from modbus the outstanding 'open' issue with it  is the full list of modbus registers as implemented by MN, can you open it?
I cannot understand why the my midnite data needs encrypting, it's seems the only people being denied access to it are the owners of the data, data security under the control of an external agency
is not really acceptable.
A suspicious (paranoid) classic owner using my midnite could be justified being concerned that their network privacy/security is compromised by permitting a foreign server to take unknown data from a device on their network.  How would they know the classic firmware is not discovering and extracting data from other network devices? Should they be concerned? Or should the data be truly open for owners to verify. 

Anyway, enough stirring, is there a modbus register that disables encryption?

Dgd

[RossW]

A suspicious (paranoid) classic owner using my midnite could be justified being concerned that their network privacy/security is compromised by permitting a foreign server to take unknown data from a device on their network. 

For the record - you are welcome to sniff the output from my code when you get it (TomW has sent you an executable that should just run).

My utility opens a connection to the classic, reads stuff (optionally writes stuff if told to) and closes the connection. It displays what you asked on the screen (or can be redirected to a file or elsewhere).

Finally, if you've told it to, it'll open a connection to one of my servers on port 80 and send a plain, unencrypted http request with a number of modbus register/value pairs in a single packet. It then waits for a single reply packet from my end which will either be "Logged OK", or other response as may be relevant. It then closes that connection and terminates.

Source isn't available at the moment because it's still (rapidly) evolving as people <fx: look towards Iowa> are powering the whole "feature-creep" cycle you expect of new code

Once you have the code there, you'll want to test it. Whatever you call the program (lets call it "classicmodbus")
type:   ./classicmodbus -p ip.of.your.classic       (ip can be either dotted-quad, or fqdn)
You should get a response back virtually instantly, along the lines of this:

% ./classicmodbus  -p `cat classic.addr` 
Solar2 Display Panel 1.0.9, RossW
State BulkMppt
Firmware 1370
ClassicTime 09:30:56  04/06/2013
  611 Watts out
50.3 Volts (Battery)
105.8 Volts (PV)
12.1 Amps (Battery)
  0.9 kWh today (17 amphours)

If that works, drop the -p and add a modbus register, like:
% ./classicmodbus `cat classic.addr`  4115
ID Solar2
ClassicTime 09:36:04  04/06/2013
4115 502 (0x1F6)

You can use ranges of addresses and/or individual address - there probably is a limit, but its more than you're likely to encounter.

% ./classicmodbus `cat classic.addr`  4115-4117 4132
ID Solar2
ClassicTime 09:37:57  04/06/2013
4115 507 (0x1FB)
4116 1067 (0x42B)
4117 187 (0xBB)
4132 246 (0xF6)

And it takes a number of primative operators. Eg, registers 5115-4116 are in tenths (507=50.7V for example), so you can do some maths directly in the program to make subsequent logging and data-handling easier.

% ./classicmodbus `cat classic.addr`  4115-4117/10 4132/10
ID Solar2
ClassicTime 09:39:45  04/06/2013
4115 51.0
4116 104.5
4117 19.6
4132 24.6

Finally, when you're ready to start logging outside... add the  -w  (write)  switch.
You may also find -d (debug) and -dd (more verbose) switches interesting.

Let me know how you get on!

[atop8918]

For the Mymidnite part of the protocol there is no register to disable the encryption. You are free to use the MODBUS protocol to extract data off of the Classic. The mymidnite protocol is simply a home-rolled MODBUS-over-http so there's no magic going on in there.

I'm not sure why the push for the http mymidnite protocol, Dgd. This is a free additional service we provide for your convenience. If you run your own server locally you can use Mango, modscan, or any number of third party tools to access the same registers directly using MODBUS.  Mymidnite can be disabled from the Classic itself if you are concerned about your data or what is being transmitted to our server. Our MODBUS register specification is available on the website, I think boB has just posted the newest revision which includes the datalogging file extraction.

We aren't trying to hide anything here or make things cryptic on purpose, I personally feel that your data is your data and you should have the option to have exclusive access to it. This is why I have made it possible to disable the mymidnite service at the Classic, why we encrypt your private data from prying eyes, and why we provide the MODBUS register specification. If we additionally released the mymidnite protocol, you would still have to write the code for the server receiving end so you're not saving any time or effort on that part.

[dgd]

I'm not sure if this is a bug or feature.
Thanks to Andrew I now have two Classics in MyMidnite
I noticed this  'feature'. If the Sliding window below main graph window is moved for the upper Classic then it moves is the main graph for the lower Classic as well
If this is a design feature, which would be  ok with me, then we do not need the sliding date window for each Classic. One would do at the bottom of the screen.
If its a bug then ok, however please consider not debugging it but  evolving it to feature status..

dgd

[atop8918]

Ah, well, spotted!

Yes it is a feature although there are bugs in it still (for instance the other slider still doesn't move properly when its companion is moved).

I did consider only one navigaotr, but then there is the issue of it scrolling off the page if you have more than 3 devices in a system. Plus which data to show in the slider?

I'm open to any and all suggestions!

[dgd]

Andrew,
I would prefer just one slider box that moved over the dates, no micro graphics in it.
Have it at the bottom of the screen, this would allow the main graph box for each Classic to be taller, the same height as the info box.
I don't think its worth the screen real estate to duplicate it for each Classic and the tiny graphics inside it.
I now find the colour code boxes inside the graph area to be annoying as they obscure graph data, I keep having to move the graph to the right to see its full height.
The scales for battery voltage I think needs to be a special case, all other go up from zero but its meaningless having battery voltage start at zero, in my case from zero to 30V for a nominal 24volt battery bank. The graph line of battery voltage getssqueezed up near the top of the graph box and has lost any meaningful visual indication of actual battery changes.
It needs a compressed scale say from 22V to 32V
The auto scaling of all the other graphed items is good
Initially there is no day selected for graph display, could this be changed to auto display the current day with last data about centre of box?
Badly need a refresh button too

And just on screen real estate - the MyMidnite window, like local app, is not size adjustable. I assume this is a 'feature' of the graphic driver you are using. But can it be made just wider? It currently uses about 60% of my screen width.
It looks like there is enough screen space to easily make the graph panel twice as wide as it is now and using the vertical space the slider box currently occupies would give us a much more meaningful (visual impact!) graph to look at.

Have thought more about boBs suggestion to dump a CSV file for text data, ok, but would it be possible to just display on screen
same as qnode14 does now?

dgd