Welcome to our peer-to-peer forums, where owners help owners. Need help now? Visit eSupport here.

English Community

ThinkStationThinkStation Workstations
All Forum Topics
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125
  • Message 1 of 8

Thinkstation P620 PCIe lane power limited?

2021-08-07, 4:12 AM

Hi I'm encountering power issues for this workstation.

 

I have a few 140W A4000 GPU which requires a 6-pin power. Ideally the PCIe lane should provide 75W whereas the 6-pin provide the 75W.

 

When I look at HWInfo, the 6-pin is pulling close to 74W whereas all the PCIe lanes are pulling a little less than 60W only. The system becomes unstable when the GPU is asking for more power than 133W it crashes (I never see it pulls 134W). 

 

When it crashes, all the PCIe lanes are still only pulling like 59.5W .. which is mindboggling. 

 

Does Lenovo limit the PCIe lanes power? 

 

Charleswg

Reply
Answer
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125

Re:Thinkstation P620 PCIe lane power limited?

2021-08-13, 7:01 AM
You could be right that Geforce and Quadro, or Turing and Ampere don't play nice with each other. I swap to all RTX A4000s and can pull 137W/140W in nvidia-smi. Though it's still a myth that HWinfo says the 6-pin is pulling 77W and the PCIE is pulling 60.6W, I have no idea how accurate they are. Thanks for the help.
Reply

Replies(7)
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125
  • Message 2 of 8

Thinkstation P620 PCIe lane power limited?

2021-08-07, 20:30 PM
I found the P620 design and it seems that the PCIE4.0 x8 slot only have 25W whereas the x16 slots have 75W. To narrow down the issues, I put all the GPU on PCIE 4.0 x16 slots. Guess what, the 12V line from PCIE x16 slots are all capped at 60W. That is in contrary to the published 75W design. Is there a bios settings that switches them ?
Reply
Options

5258 Posts

02-22-2010

United States of America

3704 Signins

51750 Page Views

  • Posts: 5258
  • Registered: ‎02-22-2010
  • Location: United States of America
  • Views: 51750
  • Message 3 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-09, 21:12 PM

Something definitely seems off with this.

 

The x16 slots on P620 are not power limiting output to the card in any way.  The spec is 75W for the slot, and up to 75W of aux 12V power for the 6 pin connection, but I don't think the spec calls out that the slow power must be used "first" so to speak.  So the 6-pin connection pulling more power than the slot is likely not a spec issue in any sense.

 

The real concern is why your system is not acting in a stable manner.  When you say the system crashes...what are the symptoms of this?  Is it a BSOD?  Is the system still powered on and just hung up?  Is it a hard power off?  If it powers off, do you need to pull AC power in order to get the system to boot properly again?

 

Depending on the answers to the questions above, I'm leaning towards this being a software crash of some sort (likely not power related at all) or a hardware problem possibly with the GPU (could be power related).  I do have some concern with exactly how HWInfo is collecting power information, especially at the aux 12V connector input, but I admittedly don't have much experience with that software.

 

If you could provide your full system configuration, as well as detail on the A4000 GPU being used (vendor/vendor PN), that would be helpful as well.  I don't think there are that many vendors currently building that GPU, but it would help to know if it was purchased 3rd party or from Lenovo directly.

 

25W from anything other than a x16 connector is in accordance with the PCIe spec from what I recall.




If we've helped resolve your issue, please be sure to mark your topic as solved!
Reply
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125
  • Message 4 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-09, 21:17 PM
Do you mean P620 instead of P720?
Reply
Options

5258 Posts

02-22-2010

United States of America

3704 Signins

51750 Page Views

  • Posts: 5258
  • Registered: ‎02-22-2010
  • Location: United States of America
  • Views: 51750
  • Message 5 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-09, 21:22 PM

Oops....yes sorry P620 :)  I'll fix that.




If we've helped resolve your issue, please be sure to mark your topic as solved!
Reply
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125
  • Message 6 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-09, 21:26 PM

Assume you mean to reply the P620 thread. 

 

Originally I have a PNY RTX A4000 on the PCIE x8 slot 6 and another PNY RTX A4000 on the PCIE x16 slot 4, and another 2080 Super in the x16 slot 1. They're all retails and not from Lenovo.

 

Both RTX A4000 are drawing 60W from the slot and close to 75W from the 6-pin. It's different from the specs published here https://thinkstation-specs.com/thinkstation/p620/ where the x8 slot is supposed to provide 25W.

 

When one of the A4000 trying to get 136W, it pull 59.5 W from the slot and over 76W from the 6-pin, causing occasional crash.  All the slots hover around 60W max (59.5W).

 

I then disabled all the x8 slots and use only x16 slots, and it's still only pulling close to 60W from the slots. 

 

On the contrary, the 2080 Super card only pull moderate amount of power from the 8-pin, 6-pin and slot (~40W) but it could be just a design difference.

 

@ psuturtle wrote:

Something definitely seems off with this.

 

The x16 slots on P720 are not power limiting output to the card in any way.  The spec is 75W for the slot, and up to 75W of aux 12V power for the 6 pin connection, but I don't think the spec calls out that the slow power must be used "first" so to speak.  So the 6-pin connection pulling more power than the slot is likely not a spec issue in any sense.

 

The real concern is why your system is not acting in a stable manner.  When you say the system crashes...what are the symptoms of this?  Is it a BSOD?  Is the system still powered on and just hung up?  Is it a hard power off?  If it powers off, do you need to pull AC power in order to get the system to boot properly again?

 

Depending on the answers to the questions above, I'm leaning towards this being a software crash of some sort (likely not power related at all) or a hardware problem possibly with the GPU (could be power related).  I do have some concern with exactly how HWInfo is collecting power information, especially at the aux 12V connector input, but I admittedly don't have much experience with that software.

 

If you could provide your full system configuration, as well as detail on the A4000 GPU being used (vendor/vendor PN), that would be helpful as well.  I don't think there are that many vendors currently building that GPU, but it would help to know if it was purchased 3rd party or from Lenovo directly.

 

25W from anything other than a x16 connector is in accordance with the PCIe spec from what I recall.

 

Reply
Options

5258 Posts

02-22-2010

United States of America

3704 Signins

51750 Page Views

  • Posts: 5258
  • Registered: ‎02-22-2010
  • Location: United States of America
  • Views: 51750
  • Message 7 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-12, 18:33 PM

Ahh OK this is helpful info.

 

Though you still didn't respond to my questions asking for more detail on symptoms of how the system is "crashing".  This is important as it might solidify the debug path and how to attack the problem.

 

I would suggest using nvidia-smi to monitor GPU power as opposed to trying to use the other tool.  I would also not get caught up on the slot power vs. aux power discussion as I think that's a major red-herring in this discussion (certainly at this point anyway).  There is no trigger/mechanism to limit the lower bandwidth slots (x1, x4, x8) to only delivering certain amounts of power (25W) on P620.  And the info that it could possibly be pulling 76W through the aux power connector is most certainly NOT what's causing your system to crash.  Again there's no trigger or mechanism to limit power in that specific manner on the aux power side either.

 

Obviously there are some concerns with the GPU combo you're using.

- Mixing GeForce and Quadro is not supported (by Lenovo or Nvidia)

- Mixing different GPU architectures, in this case Turing and Ampere, is not supported (by Lenovo or Nvidia)

 

I can't say for certain if these are contributing to your issue, but I also can't rule it out either.

 

So a couple of things to follow up with:

1.   Describe the specific symptoms of what happens when your system crashes (i.e. hard shut down, reboot, BSOD, etc.....the more detail the better here, but don't clutter it with power info at this point).

 

2.  Provide the specific vendor/model of your 2080 Super.  There are a plethora of different version of this card out there, and we'd want to check the specs for your specific card to know what we're up against.

 

3.  Try rerunning the failing scenario/workload with either just the 2080 Super installed, or with both A4000 GPUs installed.  I suspect both might pass, but it would be a good data point.




If we've helped resolve your issue, please be sure to mark your topic as solved!
Reply
Answer
Options

9 Posts

08-07-2021

United States of America

10 Signins

125 Page Views

  • Posts: 9
  • Registered: ‎08-07-2021
  • Location: United States of America
  • Views: 125
  • Message 8 of 8

Re:Thinkstation P620 PCIe lane power limited?

2021-08-13, 7:01 AM
You could be right that Geforce and Quadro, or Turing and Ampere don't play nice with each other. I swap to all RTX A4000s and can pull 137W/140W in nvidia-smi. Though it's still a myth that HWinfo says the 6-pin is pulling 77W and the PCIE is pulling 60.6W, I have no idea how accurate they are. Thanks for the help.

0 person found this solution to be helpful.

This helped me too

Reply
Forum Home

Community Guidelines

Please review our Guidelines before posting.

Learn More

Check out current deals!

Go Shop
X

Save

X

Delete

X

No, I don’t want to share ideas Yes, I agree to these terms