Welcome to our peer-to-peer forums, where owners help owners. Need help now? Visit eSupport here.

English Community

Datacenter SystemsBladeCenter / Flex Systems
All Forum Topics
Options

20 Posts

04-18-2018

New Zealand

40 Signins

245 Page Views

  • Posts: 20
  • Registered: ‎04-18-2018
  • Location: New Zealand
  • Views: 245
  • Message 1 of 11

EN4093R Failover/Failback Not Working

2018-04-30, 5:03 AM

Hi all!

 

We have been trying to configure UFP with failover enabled on our new EN4093R switches.

 

We have 2 uplinks into each switch (EXT1,EXT2), these connect to a Nexus 3K switch. 

 

When we disable all 4 ports on the Nexus, UFP failover kicks in as we expect and we see the corrosponding NIC's change to "Network cable unplugged" in the hosts OS. The issue is - when we re-enable the ports, the guest OS shows the NIC's as online again, but traffic will not flow between hosts i.e. ping.

 

To resolve the issue, we can either disable then re-enable the internal vPorts, or disable then re-enable UFP.

 

We have tried amon and mmon and get the same issue.

 

The switches and NIC's are running the latest firmware/drivers.

 

vNIC's work fine, but will not suit our application.

 

Has anyone had this problem with UFP failover/failback? Does anyone have a lab environment they would be able to test this out on?

 

 

Thanks!

Pter

Reply
Options

Re: EN4093R Failover/Failback Not Working

2018-04-30, 5:30 AM

Hi,

Can you share the EN4093R (both) config file, also what is the teaming configured on hosts, active-active / active-passive?

Both EN4093R are connected to a single upstream device?

Reply
Options

20 Posts

04-18-2018

New Zealand

40 Signins

245 Page Views

  • Posts: 20
  • Registered: ‎04-18-2018
  • Location: New Zealand
  • Views: 245
  • Message 3 of 11

Re: EN4093R Failover/Failback Not Working

2018-04-30, 7:38 AM

Hi!

 

Switch config at the bottom of this post. Both switches have identicle configs. 

 

As we're currently in the test/build phase, both EN4093R's are attached to a single Nexus 3K switch.

 

The hosts are running Server 2016 using Switch Embedded Teaming (SET) switches. However, the same results were observed when using no teaming.

 

interface port INTA1
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA2
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA3
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA4
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA5
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA6
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA7
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA8
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA9
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA10
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA11
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA12
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA13
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port INTA14
        switchport mode trunk
        switchport trunk allowed vlan 1
        exit
!
interface port EXT1
        switchport access vlan 1071
        tagpvid-ingress
        exit
!
interface port EXT2
        switchport access vlan 1071
        tagpvid-ingress
        exit
!
interface port EXT9
        switchport access vlan 1072
        tagpvid-ingress
        exit
!
interface port EXT10
        switchport access vlan 1072
        tagpvid-ingress
        exit
!
vlan 1071
        name "VLAN 1071"
        vmember INTA1.1
        vmember INTA2.1
        vmember INTA3.1
        vmember INTA4.1
        vmember INTA5.1
        vmember INTA6.1
        vmember INTA7.1
        vmember INTA8.1
        vmember INTA9.1
        vmember INTA10.1
        vmember INTA11.1
        vmember INTA12.1
        vmember INTA13.1
        vmember INTA14.1
!
vlan 1072
        name "VLAN 1072"
        vmember INTA1.2
        vmember INTA2.2
        vmember INTA3.2
        vmember INTA4.2
        vmember INTA5.2
        vmember INTA6.2
        vmember INTA7.2
        vmember INTA8.2
        vmember INTA9.2
        vmember INTA10.2
        vmember INTA11.2
        vmember INTA12.2
        vmember INTA13.2
        vmember INTA14.2
!
ufp port INTA1 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA1 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA1 enable
!
ufp port INTA2 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA2 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA2 enable
!
ufp port INTA3 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA3 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA3 enable
!
ufp port INTA4 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA4 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA4 enable
!
ufp port INTA5 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA5 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA5 enable
!
ufp port INTA6 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA6 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA6 enable
!
ufp port INTA7 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA7 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA7 enable
!
ufp port INTA8 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA8 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA8 enable
!
ufp port INTA9 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA9 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA9 enable
!
ufp port INTA10 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA10 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA10 enable
!
ufp port INTA11 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA11 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA11 enable
!
ufp port INTA12 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA12 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA12 enable
!
ufp port INTA13 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA13 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA13 enable
!
ufp port INTA14 vport 1
        network mode tunnel
        network default-vlan 1071
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA14 vport 2
        network mode tunnel
        network default-vlan 1072
        qos bandwidth min 50
        enable
        exit
!
ufp port INTA14 enable
!
ufp enable
!
portchannel 65 lacp key 1000
portchannel 66 lacp key 1001
!
!
!
spanning-tree mst configuration
        name ""
        exit
!
spanning-tree stp 55 vlan 1071
spanning-tree stp 56 vlan 1072
!
interface port EXT1
        no spanning-tree mst 0 enable
        exit
!
interface port EXT2
        no spanning-tree mst 0 enable
        exit
!
!
interface port EXT1
        lacp mode active
        lacp key 1000
!
interface port EXT2
        lacp mode active
        lacp key 1000
!
interface port EXT9
        lacp mode active
        lacp key 1001
!
interface port EXT10
        lacp mode active
        lacp key 1001
!
failover enable
failover vlan
failover trigger 1 amon admin-key 1000
failover trigger 1 enable
!
failover trigger 2 amon admin-key 1001
failover trigger 2 enable
!
!
!
!
!
!
!
cee enable
!

 

 

Pter

Reply
Options

Re: EN4093R Failover/Failback Not Working

2018-04-30, 19:10 PM

Hi,

EN4093R config is good. I do not find why this config is not working!

Looks like its time to involve lenovo for further isolation. Let us know if you found a solution.

Reply
Options

20 Posts

04-18-2018

New Zealand

40 Signins

245 Page Views

  • Posts: 20
  • Registered: ‎04-18-2018
  • Location: New Zealand
  • Views: 245
  • Message 5 of 11

Re: EN4093R Failover/Failback Not Working

2018-04-30, 20:30 PM

Hi!

 

I have tried... but they said it would need to be a paid engagment as it's "config help" that we need...

 

Does anyone have a test environment that they could test this in?

 

 

Thanks!

Reply
Options

2 Posts

11-21-2016

United Arab Emirates

6 Signins

49 Page Views

  • Posts: 2
  • Registered: ‎11-21-2016
  • Location: United Arab Emirates
  • Views: 49
  • Message 6 of 11

Re: EN4093R Failover/Failback Not Working

2018-05-10, 10:20 AM

Hello,

Were you able to solve your problem? If yes please share the details. 

If not, then..

from your config, I see that you have 2 ufp vports of each INT in use and have 2 separate uplinks & failover configured.
Also that you have enabled vlan failover in addition to amon on both uplink per switch., 

1. Can you let us know where are the EXT9-EXT10 connected (for both the swtiches)?
2. Issue is with both vports?
3. Share your cisco side portchannel & vlan configuration for both uplinks.

A rough diagram uploaded might give a better view. 

Pankaj,

Reply
Options

20 Posts

04-18-2018

New Zealand

40 Signins

245 Page Views

  • Posts: 20
  • Registered: ‎04-18-2018
  • Location: New Zealand
  • Views: 245
  • Message 7 of 11

Re: EN4093R Failover/Failback Not Working

2018-05-13, 21:27 PM

Hi Pankaj,

 

Not resolved yet :( The ticket i created with Lenovo is now with their product team as we believe that it's a driver issue.

 

I had done further testing with two Ubuntu 18.04 servers and the issue does not occur.

 

I am still yet to have someone attempt to replicate the issue, even Lenovo hasn't done this (don't know why they won't try...).

 

EXT1-EXT2 connect to a single Cisco switch.

EXT9-EXT10 are cross connects between the two EN4093R switches.

 

The same issue happens if i shut EXT9-EXT10, so not much point in providing Cisco config as this test eliminates the Cisco from the equation. 

 

You can replicate the issue 2 ways.

 

1. No host side teaming. Hyper-V role uninstalled.

2. shut EXT9-EXT10.

3. no shut EXT9-EXT10.

4. Disable and re-enable host side NIC's. Pings/traffic between hosts will not start working again, as you would expect it to.

5. shut/no shut INT ports, or disable and re-enable ufp v ports

 

1. Create host side team "New-VMSwitch -Name "SETswitch-Host-Mgmt" -NetAdapterName "Slot1 Port 1","Slot1 Port 2""

2. shut EXT9-EXT10.

3. no shut EXT9-EXT10. Pings/traffic between hosts will not start working again, as you would expect it to.

4. shut/no shut INT ports, or disable and re-enable ufp v ports

 

 

Pter

Reply
Options

200 Posts

04-23-2015

Japan

758 Signins

6570 Page Views

  • Posts: 200
  • Registered: ‎04-23-2015
  • Location: Japan
  • Views: 6570
  • Message 8 of 11

Re: EN4093R Failover/Failback Not Working

2018-05-15, 6:46 AM

Hello.

 

Do you check NIC firmware version?

Not sure what adapter/firmware/driver you use, but looks symptom is similar to following one.

 

Slow network performance in UFP mode after switch failover with EmulexCN4054 or CN4054R - Lenovo x86 Servers

https://support.lenovo.com/jp/ja/solutions/ht116283

 

Thank you.

 

Reply
Options

20 Posts

04-18-2018

New Zealand

40 Signins

245 Page Views

  • Posts: 20
  • Registered: ‎04-18-2018
  • Location: New Zealand
  • Views: 245
  • Message 9 of 11

Re: EN4093R Failover/Failback Not Working

2018-05-15, 8:03 AM
Hi!

We're running the latest drivers/firmware.

NIC's are CN4052S's.

That issue seems a bit different to ours :( We lose connectivity completely.



Pter.
Reply
Options

19 Posts

07-17-2018

United States of America

28 Signins

332 Page Views

  • Posts: 19
  • Registered: ‎07-17-2018
  • Location: United States of America
  • Views: 332
  • Message 10 of 11

Re: EN4093R Failover/Failback Not Working

2018-07-23, 16:59 PM

I did some checking internally and found that this issue was reported to our Networking PE group. This issue is resolved by a code update from Broadcom that is expected to be released at the end of October.    

Reply
Forum Home

Community Guidelines

Please review our Guidelines before posting.

Learn More

Check out current deals!

Go Shop
X

Save

X

Delete

X

No, I don’t want to share ideas Yes, I agree to these terms