Simple RS485 monitoring of Magnum Energy Inverter/Charger

[CharlesG]

I just reviewed the entire thread and, unless I missed something, I see you doing what I did when I started reading the network. My code is available at the Gitlab URL I provided earlier. I wrote mine in Java but the principles are the same in other languages.

The problem I had was finding finite records when trying the read the serial port. The Magnum protocol acts like a fire hose and just spews packets with little time in between. I too found I could not get reliable packets. My solution was to break the work into two parts.

In my application I log data every 30-60 seconds rather than maintain a full blown, full time log of packets. This allowed me to read records differently.

First I run a very tight, execution time efficient loop that just reads packets and store the result in an array. I do NO analysis at this time. Just read, store, repeat. I picked 50 as the normal count of the packets, but it is a run time parameter. 50 takes about 2 seconds and I am getting lots of packets, few malformed ones and many duplicates. But  sometimes I only get one occurrence, for example AGS generates two types of packets, the second one is infrequent. The way I read a packet is a serial read of up to 21 bytes (largest packet) into a very large buffer in case of an overrun. The read is set with timeout = 0. I loop on this, appending bytes in a packet until I get 0 bytes read. This signals I have a packet. Because this is a tight loop the code is keeping up with the fire hose. I always discard the first packet read as it's probably flawed so technically I read 51 packets but only process 50.

Now that I have the data, I can "slowly" analyze and parse the data without the need to return for more data. The net of the 50 packets is a data point. I don't worry about duplicates, i just let the latest packet overlay the earlier value. This technique is also insensitive to the devices on the network. if the device isn't there I don't get a packet. If it is there, I'll get a packet in the 2 second time span.

I have found this brute force approach provides a steady and stable set of data points. It's not the fastest for getting one packet but I'm more interested in getting complete information at one point in time. A 2 second delay is worth it. Since the number of packets collected is a run time setting, I can fine tune this as needed. A complex system with lots of devices may need a bigger sample size whereas one with just an inverter would use less.

Before I settled on this approach I tried various other techniques like measuring the delay between character reads and analyzing the packets and stopping when I had one of each. But all these routines had many bad packets. I just let the serial reader read the data, which in Java at least, it does very well, and store the result.

This may not be the ideal solution but it is working for me.

Refer to lines 118 through 133 of this program  https://gitlab.com/Magnum_Energy/distribution/blob/master/src/ca/godwin/magnum/RS485Reader.java#L118 for the read loop

[CharlesG]

I have finally developed a functioning Python based program that seems to read all know Magnum Energy data packets. It is available here.

https://pypi.org/project/pymagnum/

[Watt_]

Hello Charles

It's been a while that I have been here and had a look at the thread. Nice to come in and see progress. I will have a look and try your software.  Thanks

[CharlesG]

Please let me know what happens. good, bad or ugly.

Thanks

[Watt_]

Hello Charles

Heck of a time getting the software on my pi to spit something out. Most likely an error on my part.

I was not able to get pip of any flavor to work for me. I finally installed the build from the tar.gz file. After a bit of messing about, I got it to spit out some info. But only my router.  My wires oriented like they are, i can read  magnum data with my scripts. I haven't flipped the wires to check with yours but, here is some of the data using your py program.

Options:cleanpackets=True, device='/dev/ttyUSB0', packets=50, timeout=0.001
Length: 5 UNKNOWN   =>FC000075BC
Length: 6 UNKNOWN   =>01013316221E
Length: 5 UNKNOWN   =>7301040002
Length: 4 UNKNOWN   =>5900FF00
Length: 6 UNKNOWN   =>00DA8A643C20
Length: 5 UNKNOWN   =>0000D2BE88
Length: 6 UNKNOWN   =>0014005A0078
Length: 3 UNKNOWN   =>0478A2
Length: 3 UNKNOWN   =>000001
Length: 6 UNKNOWN   =>FB000079BB01
Length: 6 UNKNOWN   =>013316221E73
Length: 5 UNKNOWN   =>0100000259
Length: 3 UNKNOWN   =>00FF00
Length:20 UNKNOWN   =>00DA8A643C200000D2BE880014602800200A00A3
Length: 2 RTR_91    =>9120
Length:13 UNKNOWN   =>000001FB000079BB0100331622
Length: 5 UNKNOWN   =>1E73010000
Length: 4 UNKNOWN   =>025900FE
Length: 4 UNKNOWN   =>0000DA8A
Length: 6 UNKNOWN   =>643C200000D2
Length: 5 UNKNOWN   =>BE8800143C
Length: 6 UNKNOWN   =>3C00200000A4
Length: 4 UNKNOWN   =>000001FC
Length: 5 UNKNOWN   =>000079BB01
Length: 6 UNKNOWN   =>003316221E73
Packets:25 of 50 with 24 UNKNOWN, in 0.75 seconds

And the error i got...

/usr/bin/python3: Error while finding module specification for 'magnum/tools/magtest.py' (AttributeError: module 'magnum/tools/magtest' has no attribute '__path__')

[CharlesG]

I have opened an issue on the GitHub site https://github.com/CharlesGodwin/pymagnum/issues/3 relating to the errors occurring while running the test program. I did this so others, not on this forum, can see the issue and the final resolution. This will help others.

What were your pip problems. I want to improve my documentation to ensure you and others have no further problems.

I assume you followed the steps described here. https://pymagnum.readthedocs.io/en/latest/installation.html#id1

Please tell me what errors messages you got.

If you cannot, for any reason, directly reply to the GitHub issue, send an email to magnum@godwin.ca and I will post your reply.

Thanks for your patience.

[Watt_]

I have included some errors in your thread, reading rs485.

[CharlesG]

I have posted more details on GitHub
https://github.com/CharlesGodwin/pymagnum/issues/3