Lost connection to OpticsRE - Please help?

[BjornkinVT]

So last December I bought an IOGear wireless ethernet adapter and connected the Mate3s to the wifi and loved having access to the data online.

Then, in mid March, I lost the connection. It may have been related to a loss of power after several days without sun and not running the generator (manual start, and I was away) I've tried reconnecting the wifi, and it seems to be working (I can see activity lights blinking on the back of the Mate3s)

The System Settings shows an appropriate IP address and such from DHCP, but when I press the PORT button, all the information is zeroes and the OpticsRE status is Connecting.

The last recorded data on OpticsRE is:

Type System Device Port Description Initial State New State Unit User Timestamp

Status Mate3s COMM M Gateway Device is offline online offline -- -- 03/14/2023 17:55

Can anybody shed any light on this problem? Suggestions? Cautions?

The system is off-grid with a small gasoline generator that I run (occasionally a lot in the dark of winter here in Vermont)

The solar itself is working fine, it just won't talk to the internet.

Help please?

Thanks in advance!

[raysun]

The Mate3/3s ethernet module is not the most robust hardware in the industry. It is very sensitive to network failures "down the line", and has poor recovery capabilities on its own.
The first, and easiest, recovery step is to power cycle the Mate by unplugging it from the Outback Hub, waiting a moment, then plugging it back in. The ethernet processor will be reset on "cold start".
I assume you've tried that so next.
DHCP handshaking may be confused after the outage. Note the DHCP assigned values, then disable DHCP, and manually enter the network parameters. The host (Mate's) address may need to be assigned a value outside the router's DHCP address pool to avoid conflicts.
In the physical realm:
• Check the link and activity LEDs on both the Mate's ethernet port and the adapter port to which the Mate is connected. Both LEDs at both ends should show signs of life, especially during boot up.
• Replace the Mate to Adapter ethernet cable with a known good Cat5/6 "straight through" ethernet cable. (If you don't have spares, borrow one from another piece of ethernet-enabled equipment. DO NOT use one of the Outback "green" cables. They're RJ45 but not Cat5/6 compliant.)
• Test the I/O Gear adapter with a laptop or other ethernet-enabled device.
• Bypass the I/O Gear adapter by (temporarily) connecting the Mate directly to the LAN's main router (with presumably a looong ethernet cable of less than 300').
If the hardware checks yield no joy, try updating the firmware, using the latest version. It may correct the corrupt ethernet code.
MATE3s Firmware Updates - OutBack Power Inc https://www.outbackpower.com/resources- ... re-updates

Good luck!

[BjornkinVT]

Thank you, Raysun!

I did try the power cycling by unplugging the CAT5 cable from the back of the Mate3s, with no change in behavior.

What DID work was turning off DHCP. The Mate3s remembered the settings, probably because I took note of the settings. ;-{)

It took a while, but the connection to OpticsRE is working again!

My only regret is that I didn't fix it sooner, and lost 2 months worth of data, in the transition from Winter to Summer. :/

Thanks again!

[dtich]

Hi all. Having this issue with the ethernet and satellite internet. When the satellite goes down, which it does from time to time, the MATE3s doesn't reconnect and a power cycle is necessary. That solves the issue.

My question is: does manually coding the IP solve or help this? Is this primarily a DHCP issue? I didn't know this. If I had I would have manually configured it from the start, as I do for most of my networks. I left the MATE in DHCP because I (foolishly?) thought that would be MORE robust for the remote system.

I cannot easily get to the MATE now to reconfigure but I can get it done if it would make a difference. As it is now, I go down and require a power cycle about once a week. Which is ridiculous.

Are there any thoughts on this, or any other solutions that people can share?

I got this, finally, from OB tech support, after many back and forths with nothing of use being offered on their side. Then someone decided to say this:

Hello,

The satellite internet issues have been a known problem for a long time, and I’m not sure when if ever, engineering is going to commit to fixing them. There is new firmware development happening, but I do not know if they will be able to backport those changes to the existing Mate3 or Mate3s.


At this point, because the system is remote, I’d advise you to get some additional hardware to reset the power connection remotely. There are a variety of devices you can buy to do this, but it will involve splicing the cable open so that the correct power cable can be hooked up to a device that can remotely control the power delivery to the Mate3s.

Some customers on the outback power forums have discussed these setups.

Have to say, this is pretty lame. If a fw fix could help this, I can't see how the engineers wouldn't be prioritizing it. These systems aren't THAT complex honestly, that they are loathe to do a fw fix on this is... odd, to say the least. Is this a hardware issue? Somehow.. Although, in writing that I just don't see how this isn't addressable in fw. Ethernet connections/IP sockets are about as fundamental a network component as you can get, how is it SO fragile on a REMOTE MONITORING SYSTEM? It would be sort of laughable if it weren't so painful.

Any thoughts appreciated. Thanks!

[dtich]

Was just in the shower thinking about this and I was mulling over the fact that my home network has, oh, 250 ethernet ports or so, and there are internet failures and power outages from time to time, and not ONE of those ports fails to reconnect. In fact, most power outages go by unnoticed because everything comes back up and running without a squawk and I only know because my automation system tells me it happened.

And at work the network is..... huge. Thousands of nodes. Across cities. Countries. Oceans. And that stuff works. Mostly, lol.

I'm sure there's a good reason, perhaps it is just a bad ethernet controller, as in, not a good one, but... how odd.

Anyway. A little lost as to why this is an unfixable problem. Thanks.

[dtich]

ok. i am calmer now. apologies.

two real questions: does a fixed IP help with this issue (ethernet not recovering from down wan port) at all?

and, i have an sd card installed, and logging enabled for 60s, 120 days retention, with the hopes that this data would backfill during those times when optics is not reachable (such as every night when the internet goes off between 2 and 6am).

the data from the overnight down period has not been filling in. well, it did for the one night right after i installed the card, but not since then. settings did not change. is there something, a setting, that i need to check/enable in order for this data, any data, from times when the internet is down, to backfill and upload to optics when service resumes?

thanks!

[raysun]

A fixed IP address helps only in so far as it removes the complexity of DHCP handshaking from the equation. It won't, of itself, make the Mate more robust in recovering from a network disconnect.
The data logged to the SD card is supposed to automatically replay on reconnection (there's a series of .RPY files that are to be read.) In practice, it rarely fully recovers the data into Optics of its own accord. If the outage is too long, some of the RPY files get overwritten. Other file corruption keeps it from being read in other instances. At times, the recovery routine simply lumps the recovered data into a single hour.
As mentioned, the Mate doesn't recover from network outages as effectively as other network attached devices. However, the Mate isn't the source of the issue, the network is. My system was previously connected to the internet via a radio link, and periodic network disruptions caused the Mate to "go dark" with no recourse but manual restart. When fiber optics broadband came available, I switched. In the past two years I've had no Mate disconnect issues. (Knock on thermoplastic.)
I'm not excusing the Mate. It's ethernet processor is the source of much frustration. However, it's such a long-standing problem, if OB was going to fix it, they surely would have done so by now. We've been left with devising work-arounds. In my case, I simply don't rely on long-term data from OpticsRE. I tend to look at day-to-day operations and call it good. If I couldn't go to sleep at night without calculating rolling annual performance data, and graphing it for study with my bedtime Ovaltine, then I'd use a local data collection and monitoring tool. One stands out: Andrew Welch's WattPlot ACKcess 6.0. It does everything Optics does, and a whole lot more. It runs locally on a PC, so the Mate need not even brave the cold, cruel Internet in order to be a contributing member of society. www.wattplot.com

[dtich]

thanks for the response raysun, thoughtful and thorough as always.

yes, i hear you on the dhcp, i will likely leave on as i don't think that has any bearing on any of the issues.

i am still at a loss to understand what aspect of a satellite internet connection makes the ethernet port so sketchy, but i hear that the empirical experience seems to point that way.

i doubt i will hook up any sort of hardware restart system as honestly monitoring isn't mission critical, the system works fine even if i don't know it is :D someday it would be very nice to have optics be more reliable long term.

wattplot looks like it was cool and helpful, but i think i'm past that. no need to set that up, if i have access to the data i can easily graph myself, it's about access to the data not interpreting it. BUT, thanks a lot, all very helpful.

[raysun]

One glimmer of hope for improvement is one of the OB software engineers popped up on the forum recently and hinted hr was working on satellite communications issue specifically. Without any details one can only speculate, but I'm assuming this will be a patch to run on the existing ethernet processor (rather than new hardware.)

[dtich]

fingers x'd..

[intellact]

wattplot looks like it was cool and helpful, but i think i'm past that. no need to set that up, if i have access to the data i can easily graph myself, it's about access to the data not interpreting it. BUT, thanks a lot, all very helpful.

Just to add a small observation here...

Among the many tools in WattPlot is the ability to grab data from the SD Card, store it locally, and provide a summary spreadsheet as an added bonus. (I have found that the AXS Port supports FTP, so this can be done automatically at regular intervals, avoiding the loss of data from overwrites. Last time I checked the MATE3, it did not support FTP - I've never had a chance to try it on a MATE3S.)

When WattPlot is monitoring, it is also creating its own version of the standard MATE data files, stored locally, so you always have that data capture too.

There's a reason that the latest production release of WattPlot is called WattPlot ACKcess.

[jonathanpr]

I was hoping this thread might help with my OpticsRE connection issue.
I just installed my new Mate3S, which is a replacement for one that burned out early this year. Everything else is identical. I have been using a Mate3SL (no ethernet connection) for the past six months or so. Fortunately, I had saved my configuration and printed the details from OpticsRE, so I was able to set up the system with both this unit and now my new 3S
So, I installed the new Mate3S today, made sure that the Firmware is updated and properly installed, loaded and confirmed my previous configuration, and then checked OpticsRE. There is no data showing.
The unit has an IP address and everything is working normally, with the ethernet settings all as expected.
I can see the system over my WiFi using its IP address, so I know it is connected, functioning and reporting.
But nothing on OpticsRE.
Am I missing something? Does OpticsRE not like the new Mate3S (maybe there is a security check using the serial number?)?
Thank you.

[dtich]

IDK, but you may have to setup a new system (on Optics) if the Mate is replaced; I believe the Optics interface IDs the system via the Mate serial #. Not sure, but might be something like this.

Post the solution please, when you find it. Good luck.

[raysun]

I was hoping this thread might help with my OpticsRE connection issue.
I just installed my new Mate3S, which is a replacement for one that burned out early this year. Everything else is identical. I have been using a Mate3SL (no ethernet connection) for the past six months or so. Fortunately, I had saved my configuration and printed the details from OpticsRE, so I was able to set up the system with both this unit and now my new 3S
So, I installed the new Mate3S today, made sure that the Firmware is updated and properly installed, loaded and confirmed my previous configuration, and then checked OpticsRE. There is no data showing.
The unit has an IP address and everything is working normally, with the ethernet settings all as expected.
I can see the system over my WiFi using its IP address, so I know it is connected, functioning and reporting.
But nothing on OpticsRE.
Am I missing something? Does OpticsRE not like the new Mate3S (maybe there is a security check using the serial number?)?
Thank you.

OpticsRE uses the Mate's MAC address to link the system to data. When the new Mate was installed and registered with OpticsRE, it initiated a new, empty data set.
You can contact Outback Technical Support and request they manually link your new Mate's MAC address with the prior installation's data set.

[jonathanpr]

Thank you. Good information to know.

[The Electron]

I lack the vocabulary and writing skills to properly communicate the level of disappointment with Outback FIXING this AWFUL ethernet processor problem in the Mate3's.