Hi all,
I have a datalogger running which retrieves some values over modbus every 10 seconds. After leaving it running for a day or two, the classic stops responding to modbus requests. Same with the localapp, it responds with "Busy" when trying to connect to the classic.
I can still make TCP connections to port 502, it just doesn't respond to modbus requests. Have to turn the controller off and on again to get it back up and running.
Any ideas?
I have forwarded this to Andrew our Guru Engineer on the App and Networking the Classic. This is something we have seen on a few rare occasions but only to certain networks. That is to say if I was to take your Classic here it would be fine and if you took mine it would do this. The Problem is for sure in the Classic but it has been a little stubborn to chase down do to the lack of it showing in engineering.
Ryan
Can you try connecting the ethernet directly up to your computer/laptop/pc instead of
a router and see if this still happens ?? Or, is this already connected that way ?
This may be important information.
boB
Yep it's currently connected to an ethernet switch. I'll connect it directly to the logger and leave it running for the day tomorrow and see what happens.
Sticky problem, but I am sure the answer is there....
I use Modpoll and bash scripting to pull in the data via a router (TCP). It has been running continuous for about three weeks. The crontab runs once a minute and gets values from three registers. The app. is off.
I did have to adjust the time on the Classic (up to 2012) this week, not sure why? Modpoll did not care, just kept on pumping out data.
hang in there,
steve
http://sunenr.homedns.org/cgi-bin/modclsc150.pl
Quote from: rosebudd on September 23, 2012, 04:22:21 PM
Sticky problem, but I am sure the answer is there....
I use Modpoll and bash scripting to pull in the data via a router (TCP). It has been running continuous for about three weeks. The crontab runs once a minute and gets values from three registers. The app. is off.
I did have to adjust the time on the Classic (up to 2012) this week, not sure why? Modpoll did not care, just kept on pumping out data.
hang in there,
steve
http://sunenr.homedns.org/cgi-bin/modclsc150.pl
awesome page you have there !!
Yes, I'm sure that the answer is just a few bytes away !
boB
....teach me not to boast..
The script stopped this morning, not sure why. Bash said "timed out" ?
I shut down the little pc, then the Classic and pc together for a minute or so. Rebooted both and all is back on....I did wiggle the router cables...(?) Could have been a connection...check the simple things first.
back in the saddle, all running again,
fog lifted in eastern Canada, sun is back on,
steve
Quote from: rosebudd on September 24, 2012, 09:45:09 AM
....teach me not to boast..
The script stopped this morning, not sure why. Bash said "timed out" ?
I shut down the little pc, then the Classic and pc together for a minute or so. Rebooted both and all is back on....I did wiggle the router cables...(?) Could have been a connection...check the simple things first.
back in the saddle, all running again,
fog lifted in eastern Canada, sun is back on,
steve
OK, just keep watching it and report back weird things.
No fog here... Just smoke from forest fires.
Other than that in eastern Washington, nice day here in Arlington.
boB
boB, stephendv,
little update,
Modbus is dropping out about every 12-13 hours. A quick on/off of the CC fixes it.
The script says "Reply time-out!"
Modpoll/Bash is reading two registers, once a minute.
I did not update the firmware's last release...that may help? Just a little bug somewhere, software is like that.
tks,
steve
Quote from: rosebudd on September 26, 2012, 08:35:33 AM
boB, stephendv,
little update,
Modbus is dropping out about every 12-13 hours. A quick on/off of the CC fixes it.
The script says "Reply time-out!"
Modpoll/Bash is reading two registers, once a minute.
I did not update the firmware's last release...that may help? Just a little bug somewhere, software is like that.
tks,
steve
Hi steve, yep that's exactly what I'm seeing, and I'm doing something very similar, cron job every minute to grab 3 values. Unfortunately "a quick on-off" is a 500m hike down to where my array is.
I have the latest firmware and am still seeing this. Have you tried boB's suggestion of connecting directly to the classic without a switch in between? (I haven't yet).
stephen,
I have a slight advantage re the 500 mile jaunt, my gear is in the basement.....but the software is challenging (for me)...and fun.
I am using TCP via a Linksys router. Suspect the bug is in the CC, because it ran for ~3 weeks flawlessly...then started to hiccup? Per boB, I am watching and experimenting. Bypassing the router is an idea, but may try a couple easier things first.
Thinking of slowing down the crontab to see if that changes the drop interval? Is it time or a number of executions?
will share observations, the Classic works great,
steve
ps logging helps with trouble shooting.
FYI, We are working on the TCP/IP issue here.
Also, you are both talking about modbus over TCP/IP, right ? And not the RS-232 modbus ?
I have not heard of any issues with the RS-232 operation which is also modbus.
Instead of powering down and on again is to go to TWEAKS, MORE, MORE2 and set A-RST
to ON.
Then, go to LOGS, DAILY and then hold down the LEFT-ARROW key and tap the ENTER key.
This will force a new day, save the logging information and then reboot the Classic because
A-RST is set.
If you pressed enter after turning A-RST to ON, the Classic will remember this
and reset itself every night at midnight. I realize this will not help if it is in the middle
of the day and you wanted to see or control the Classic remotely, but at least if you
do have to walk up to the Classic, it won't lose what power production it has made
that day as far as data logging goes. The 5 or 10 minute data will of course
be logged in either case.
Now, if there were only a way to do this remotely after it stopped talking.
Anyhow, we're working on fixing this TCP/IP stack issue.
Thanks,
boB
thanks boB,
Yes, I am using TCP/IP for communications to the Classic.
I followed your instructions and a quick reboot....up and running again.
at the bottom of the page, the month graph shows what is happening:
http://sunenr.homedns.org/cgi-bin/modbattvlt.pl
If I can help troubleshoot at this end, let me know.....
steve
Thanks boB, I've made those changes last night and so far it's been running without a problem (10 hours and counting).
Yes, I've only ever used modbus tcp because it's easier to use it remotely over my wifi.
Steve, you might be interested in cosm.com, a free data logging service, saves generating and hosting your own graphs ;)
https://cosm.com/feeds/75889
amigos,
Waited to see how long Modbus would stay up.....made it til 14.5 hours.
I doubled the time between samples, to see if it changes anything?
interesting stuff,
stephendv..thanks for the link
steve
Assuming the controller did a reset at 12 last night, mine's been going for 17 hours now. 1 minute samples.
Quote from: stephendv on September 27, 2012, 02:22:26 AM
Thanks boB, I've made those changes last night and so far it's been running without a problem (10 hours and counting).
Yes, I've only ever used modbus tcp because it's easier to use it remotely over my wifi.
Steve, you might be interested in cosm.com, a free data logging service, saves generating and hosting your own graphs ;)
https://cosm.com/feeds/75889
Yes, I noticed that site you use.
One of our engineers here used to use pachube (.com ?) that
evidently turned into cosm.com
Here is another similar type of site you may be interested in...
http://pvoutput.org/outputs.jsp
boB
Instead of powering down and on again is to go to TWEAKS, MORE, MORE2 and set A-RST
to ON.
This has effectively "fixed" the issue for me, been running for more than 36 hours now without a problem :)
Quote from: stephendv on September 28, 2012, 06:30:52 AM
Instead of powering down and on again is to go to TWEAKS, MORE, MORE2 and set A-RST
to ON.
This has effectively "fixed" the issue for me, been running for more than 36 hours now without a problem :)
Scratch that, it just fell over. I get a timeout message when trying to connect to the classic. Let's see if the reset tonight will fix it.
stephen,
I did change the sample rate to once every two minutes....that almost doubled the modbus run time...to just under 24 hours..
Trying for every three minutes, see what that looks like?
gather info,
steve
Very good observations and very useful information I think.
Thank you for this.
Stephen, I'm sure that it will fix itself tonight, but the A-RST won't
help you if the communications stopped during the day.
We are working on a permanent solution of course.
Rosebudd, that is also very good information ! Thank you both !
boB
Hey guys... What OS are you using ??
Windoze or Linux ??? Something else ???
Also, are you both using our local app ??? OR, does the classic drop with your own modbus over TCP/IP app ??
boB
boB,
I am using Linux (11.1) and your App. is shut off. My understanding is they both can't run at the same time? I have not tried the App. since starting the modbus.
I use modbus software called Modpoll and a bash script to talk to the Classic (TCP/IP). The Classic is connected thru a Linksys router to a small pc (atom ITX).
tks,
steve, alias rosebud.
ps: the 3 min. sample rate made ragged RRD graphs, back to 2 mins., for now.
I'm using linux too and the phpmodbus library. So I guess your next question is whether it still stops if we're using the local app and not our own modbus... I'll start running it now to see what happens. :)
Which reminds me of another issue: a sure way of locking it up is to connect with 2 different modbus requests at the same time. This I've definitely seen with the local app, where local app is running and as soon as I try to connect with my modbus too the classic stops responding to both and needs a restart.
Just ran the localapp only, and it took a few hours before the classic's modbus locked up and stopped responding.
this is a first...?
Modbus stopped after ~24 hours of operation,stayed off for 9 hours, then re-started again....I was away so I did not cycle the power on the Classic. It started again on it's own.
This may come under the weird category?
all fun,
steve
Quote from: rosebudd on September 29, 2012, 08:28:09 PM
this is a first...?
Modbus stopped after ~24 hours of operation,stayed off for 9 hours, then re-started again....I was away so I did not cycle the power on the Classic. It started again on it's own.
This may come under the weird category?
all fun,
steve
Are you continuously accessing the Classic or are you letting it sit for a while and coming back to it ??
My Classic here at home has not crashed at all except in DHCP mode, I see once in a while where my router
will give it a new TCP/IP address in descending numbers. like, 192.168.1.38, then 192.168.1.37, 192.168.1.36,...
but only when I write to it.
I just tried modpoll while the local app was talking to the Classic, (STATIC IP), and it would not crash.
Just keeps going. Trying harder...
BTW, the modbus program I usually use is modscan but modpoll is kind of a neat freebie program.
Think I'll see if I can compile it now so I can make it loop, etc.
boB
PS. I also would like to change modpoll so I can read a 16 bit int and not just 32 bit (2 registers)
PPS... I found the include files on a fieldpoint web site but looks like it isn't going to compile using old VC 6.0 directly anyway.
PPPPPS...
I am looping modpoll, reading many registers as fast as it will go while running the local app. Once in a while
the Local app will stop because modpoll will take over for a moment, and then the Local App comes back to
life on its own. I set Aux 1 and/or aux 2 to Toggle Test so I can immediately tell if the Local app has got the
connection or I will see the DOS window of modpoll streaming the register contents by.
It won't crash for me. This is one reason why it is kind of hard to catch this problem.
I know that our network guy can duplicate this so it will get fixed fairly soon I hope.
boB,
You are getting close to the "ah-ha" moment.
Yes, I am using the DHCP mode.
The crontab runs currently, once every two minutes...
I have not tried to write to the CC, don't want to mess anything up. The App. is still off.
The router comment may be a factor? I did test a Raspberry Pi connected to the router about the same time this started? ....one time event.
I had been reading three registers, without a problem, but then dropped off one to see if things improved. ....no change.
Things have been good for 11.5 hours since it re started by itself.
thanks for you efforts and insight,
steve
Quote from: rosebudd on September 30, 2012, 07:24:02 AM
boB,
You are getting close to the "ah-ha" moment.
Yes, I am using the DHCP mode.
OK... Since the Classic has an IP address given to it from the router by DHCP now,
change the Classic to STATIC and see what happens.
One problem I had with DHCP was that it would sometimes change address on me
for no apparent reason. Even though I think I needed to write to it to get it to
do this, maybe there is something to it for you, too.
With the latest software, you may need to write the password to the Classic to change
registers anyway... Except for the password of course.
The absolute latest firmware does not require a password if the LOCK jumper is in place.
I'm not sure if that software is released or not ?? If not, it will be in the next one coming up
in a week or less.
boB
boB,
Switched the connection over to STATIC, all is well.
I tried running the App in DHCP and STATIC mode, it would run but Modbus stopped. Bash said "TCP/IP connection was closed by remote peer!". Competition for port 502?
I am using the second last firmware update.
I held back on updating because I guessed I would lose the App function under Linux. Hoping to get Modbus working reliably and then keep updating the CC firmware.
Plan to keep monitoring and reporting things back.
One step at a time,
steve
ps just before the switch to Static, modbus had run for 12 hours, down for 2, then back up..by itself.
Rosebudd, just stick with the firmware you have. The network portion should be the same
as what we are presently shipping I think.
Thanks !
boB
Mine is running on a static IP address, always has been.
Since the classic can only talk to 1 device at a time, perhaps when it stops responding, it's actually still trying to talk over the previous connection? Both steve's and my logger make a new TCP connection every minute (or 2 minutes) to grab the modbus values, it could be that one of the previous TCP connections haven't closed properly so the classic doesn't respond to any new requests. The default tcp keepalive timeout on linux is 2 hours, so I've just changed this to 5 minutes to see if it makes a difference...
Looking good so far! I turned off the midnight reset on the classic yesterday, and with the new TCP keepalive timeout of 5 minutes on the linux box, the logger has been running every minute for 24 hours no problems. Fingers crossed.
Quote from: stephendv on October 02, 2012, 02:58:19 AM
Looking good so far! I turned off the midnight reset on the classic yesterday, and with the new TCP keepalive timeout of 5 minutes on the linux box, the logger has been running every minute for 24 hours no problems. Fingers crossed.
Interesting !
It appears to not disconnect gracefully some times.
boB
boB,
Good news at my end also...
Modbpoll has been running for three days, without a hitch, since I switched over to STATIC.
I inadvertently mislead you on how many registers I read....really, the way it is written, I only read one at a time. Kinda clunky programming, but it works for now.
many thanks,
steve
6 days later and mine is still ticking away, no connection problems :D
Steve, have you by any chance tried writing to any of the modbus values? Would be interested if you got it right! http://midnitesolar.com/smf_forum/index.php?topic=721.0
Quote from: stephendv on October 07, 2012, 04:46:35 AM
6 days later and mine is still ticking away, no connection problems :D
Steve, have you by any chance tried writing to any of the modbus values? Would be interested if you got it right! http://midnitesolar.com/smf_forum/index.php?topic=721.0
Stephen, are you doing a modbus write multiple register command with one register or write single register command ?
boB
Write multiple registers command using just 1 register.
To try and set the absorb voltage I'm writing 580 to register 4148. (Reading 4148 returns 582 which is what i currently have it set as).
To force EQ I'm writing 0x80 to register 4159.
Quote from: stephendv on October 07, 2012, 04:08:03 PM
Write multiple registers command using just 1 register.
To try and set the absorb voltage I'm writing 580 to register 4148. (Reading 4148 returns 582 which is what i currently have it set as).
To force EQ I'm writing 0x80 to register 4159.
Ah! I think I know what it is!
The latest classic code requires a password which is the serial number and not including the. Leading zeros. I'm sure that is not in our documentation as that is very new.
However, with the absolute latest firmware, you should be able to bypass the serial number password by installing the jumper on the lock jumper behind the main terminal block.
That's gotta be it.
Thanks bob! I'll give that a try. I did try sending the serial number first, but now I see that the library was disconnecting the connection between sending the serial number and sending the writeRegister command.
Will try the jumper method instead.
No dice :(
I installed the lock jumper, then turned the classic on again- but am still getting the "illegal function" error when trying to write. I'm using firmware 6-29-12.
media@media:~$ modbus read 192.168.0.16 %MW4148 1
%MW4148 581
media@media:~$ modbus write 192.168.0.16 %MW4148 580
/var/lib/gems/1.8/gems/rmodbus-1.1.5/lib/rmodbus/slave.rb:262:in `query': The function code received in the query is not an allowable action for the server (ModBus::Errors::IllegalFunction)
Strange, setting the lock jumper seems to have had the opposite effect, now I can't change any settings through the local app either.
Quote from: stephendv on October 09, 2012, 12:30:48 PM
Strange, setting the lock jumper seems to have had the opposite effect, now I can't change any settings through the local app either.
What is your exact Classic version and date again ??? We were changing that lock jumper definition around some
just recently. Originally, when the jumper was installed, it was supposed to keep ANY changes from the TCP/IP
port from happening.... THEN, since some customers do not want to have to send that password/serial number
every time a Classic boots, we reversed the logic. You may have an older version.
The version I posted should have the logic reversed to the way you want it. Jumper = UNlocked...
It's a semi-beta version of code but you should probably try it. You can always go back to an older
version if there is a problem.
http://fusion.midnitesolar.com/MidNiteSolarSetup_10-2-2012_v3.01.exe
boB
You beauty!
With the new firmware and the jumper connected, modbus writes work a charm ;D
Some notes: Previously was using the latest released firmware 6-29-12 which had the jumper-connect = no writing whatsoever setting.
Also think there's a bug with the new firmware, did the VMM then set the charging parameters, but when I checked the settings again, it had somehow put the absorb voltage at the EQ voltage.
I've got some programming to do ;D ;D ;D
Are you sure it did not do the other way around and maybe haveto set the equalization voltage to the absorb voltage?
Quote from: boB on October 10, 2012, 03:08:41 PM
Are you sure it did not do the other way around and maybe haveto set the equalization voltage to the absorb voltage?
Well I was pretty sure because the absorb voltage was set to 62.4 which would be suicidal.... but then I tried the process again this morning, VMM then set the voltages and it worked fine, everything as it should be. Where's the shrug-shoulders icon when you need it.