02-09-2019 03:42 PM - edited 02-09-2019 03:48 PM
I'm having issues with understanding the NICs on the system in question.
Namely, when changing the NIC advanced settings, the physical adapter would suddenly go to Cable Unplugged state. I've had this happen before when setting the Packed Direct option and tonight it happed while setting up the Flow Control on NIC#2.
It is a clustered system, but I'm not sure it has anything to do with that.
After it goes to Cable Unplugged, there's nothing I can do. Once it recovered of its own will, the second time I realized FW wasn't updated so I updated that and the nework driver. At the moment, everything is fully updated and it's still happening. As I'm typing this, the adapter recovered of its own volition and is now back to normally connected. It took some 30 minutes to change states.
The "other side" is a 10G copper connection and I've tried to manually set the speed so it's not on Auto negotiate but that didn't solve it. Also, I've set the Flow control on the second host and on the second NIC on this host without issue. Only this one crashed.
I'm at a loss...
EDIT: It recovered and 5 minutes later went back to Cable Unplugged. That's to say - it's been working fine for days and only after I started messing with Advanced settings did it start acting like this.
Solved! Go to Solution.
02-11-2019 06:48 AM
Welcome to the community!
Can you please provide us with more information about this system? What model NIC is installed? Do you have a similar server that is not experiencing this issue? What OS is being used?
If you have an Extra NIC laying around could you try replacing the "faulty" one and see if the behavior changes.
02-11-2019 07:20 AM
The system in question is a ThinkSystem SR530 7x08.
The NICs are Intel X722 10G.
Operating System is WS2016.
The drivers are installed from the Lenovo XClarity Updater app (though I have tried a newer version taken from Intel, too, with no luck)
We do have another system set up identically and it's working fine (for now).
Firmware has been fully updated, however there is something odd going on:
When I perform an XClarity Update Packages scan, it indicates that there is an Intel v23.2 firmware update to version 1.1937.0, but if I attempt to download it, an error occurs, saying the package cannot be found.
If, however, I perform an Individual packet scan, it will indicate that the latest firmware is installed, with version number 1.1892.0 and it does not have an update available.
That aside, the NIC also very often goes to "Connected" state, only to disconnect within a second and occasionally it fully connects, but can't receive a DHCP address. A few times it was working normally for a few days, only to crash and not wake up for a prolonged period.
Currently it's in "Network Cable Unplugged" state, attempting reconnects continuously. The driver used is the one recommended by the update utility:
At the moment, we do not have a reserve NIC around to test it.
We did test something else, which may be indicative:
Without teaming and with default settings. Just one adapter on each side, connected to a vSwitch.
When using the 10G adapter as a HyperV external v-switch connection point, we couldn't even get 1G speed between VM's on the two hosts in question. That's unclusterd hosts, just two test machines (SR530), each running one VM with 6 logical cores.
When using the 1G adapter as a HyperV external v-switch connection point, we get 1G between the VM's.
It seems there is something royally messed up with the 10G adapter and I'm starting to think it may be affecting Hyper-V as well.
Nota bene: transfer rates between the working NICs host-to-host are 10G, making it an even more of a conundrum.
02-11-2019 07:40 AM
02-11-2019 07:52 AM
Based on the symptoms it does seem like a NIC issue. Would you please try opening a Support ticket to see if this can be resolved?
02-20-2019 09:05 AM
Wanted to check-in and see if you were able to find a resolution to your port issue. If so, please share with us so it can be helpful to someone else should they run into the same issue.
02-21-2019 12:26 AM
we contacted support as per your advice and they have replaced the network cards in both of the servers. They seem to be running smoothly now.
One thing you may want to convey to your "development" guys is that the XClarity Update Express continuously offers a 1.1937 update for the NIC firmware. However, it's impossible to download it via packages or individual updates.
It's possible to download it manually by creating a local repository, but installing it is impossible, since the update always fails.
Thanks for the assistance!