cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
cmltit
Paper Tape
Posts: 2
Registered: ‎01-22-2018
Location: DE
Views: 2,786
Message 1 of 9

Stuck with buggy micro code after recent BIOS update

We upgraded a bunch of T470s notebooks with the now withdrawn BIOS update (1.21) that included micro code updates to address the Spectre/Meltdown vulnerabilities. The devices (running Windows 10) now have intermittent hangs and are almost unbearably slow to work with, so we decided to roll back to 1.20. Turns out, that the micro code updates cannot be reversed by downgrading the BIOS. So we are now stuck with almost unusable notebooks until there will be a new update. What are we supposed to do now?

aUserHasNoName
802.11n
Posts: 179
Registered: ‎06-26-2016
Location: DE
Views: 2,753
Message 2 of 9

Re: Stuck with buggy micro code after recent BIOS update

How do you know microcode was not downgraded?  Microcode is part of the FW so it should downgrade.

Did you load setup defaults after FW downgrade?
Did you try to downgrade to an older FW version 1.19 or 1.17  on one of your devices? 

 

<1.19>
 UEFI: 1.19 / ECP: 1.17
- (New) Updated the CPU microcode.

<1.17>
 UEFI: 1.17 / ECP: 1.16
- [Important] Update includes some security fixes.
              (Note)
              If the UEFI BIOS has been updated to version 1.17 or higher,
              it is no longer able to roll back to the version before 1.17
              for security improvement.

what to learn from this fiasco:

1. postpone flashing new FW until update is available on Lenovo System Update.

2.  Upgrade one device only  and do extensive tests for several days.

3.  upgrade remaining devices if tests did not show any issues.

 

 

 

przemek1234pl
802.11n
Posts: 152
Registered: ‎11-28-2017
Location: PL
Views: 2,735
Message 3 of 9

Re: Stuck with buggy micro code after recent BIOS update

There is info in security advisory on Lenovo webpage that once updated microcode cannot be reversed other way than replacing motherboard. So the only solution is to wait for better microcode which should arrive soon.

cmltit
Paper Tape
Posts: 2
Registered: ‎01-22-2018
Location: DE
Views: 2,734
Message 4 of 9

Re: Stuck with buggy micro code after recent BIOS update

Thank you for your quick reply.

I believe the code is not downgraded because the SpeculationControl script from Microsoft still shows hardware support for CVE-2017-5715. Also, the Lenovo Security Advisory LEN-18282 showed a back-flash recommendation after the updates have been withdrawn, but later removed it while commenting "MCU microcode updates cannot be reversed". You can still see that in the revision history.

Unfortunately, the older BIOS revisions also do not seem to roll back the microcode updates.

Regarding your points, this is why we updated only a couple devices (not a bunch as mentioned above, which was wrong). To be exact, 4 devices that have been actively used to evaluate the update.
Guru
Posts: 612
Registered: ‎12-30-2017
Location: PL
Views: 2,720
Message 5 of 9

Re: Stuck with buggy micro code after recent BIOS update

CPU microcode isn't downgraded, because it's not possible to do so. The only supported opertation is to update to newer version. 

 

You can do one thing though. I've found, that when I disabled Meltdown fix, my devices started to work as before update. No BSOD so far. I've used guide below:

https://winaero.com/blog/disable-meltdown-fix-amd-cpus-installing-kb4056892/

------
...
DenizOezmen
Punch Card
Posts: 5
Registered: ‎09-27-2008
Location: DE
Views: 2,713
Message 6 of 9

Re: Stuck with buggy micro code after recent BIOS update

The interesting thing here is: In theory, it should be possible to downgrade the microcode version. The reason for the blocked downgrade path seems to be the flashing program (WinFlash) used for the BIOS update procedure: The program has a special switch "/vcpu" that enables microcode updating.

 

According to the WinFlash documentation, the program evaluates three sources to determine the microcode version to be inserted into the BIOS:

 

  • the BIOS file to be flashed
  • an optional microcode file (that can be passed on the command line to the /vcpu switch)
  • the current content of the system EEPROM

 

WinFlash then chooses the most recent microcode version found in these sources. Since the EEPROM content itself is part of the evaluation, the system's microcode will never be downgraded.

 

Since microcode updates are not persistent across reboots anyway, I don't currently see a real technical reason for this behaviour, though. (It's not like there was a danger of "downgrading" your CPU into an unsupported state.) If I had to guess, I'd assume this was done for convenience reasons. Anyone with more knowledge on the matter is free to correct this assumption, though ...

 

[edited to fix formatting]

CrypticLottery
Token Ring
Posts: 51
Registered: ‎01-08-2018
Location: US
Views: 2,658
Message 7 of 9

Re: Stuck with buggy micro code after recent BIOS update

You can follow the steps listed here https://forums.lenovo.com/t5/ThinkPad-T400-T500-and-newer-T/KB4056892-multiple-problems-on-T440s/m-p... to disable the Microsoft patch for CVE-2017-5715. This is the one that can be partially fix through a microcode update.

 

It has been observed by multiple people that disabing the Windows Update for 5715 restores system stability, albit keeping you vulnerable to the flaw.

 

Hope this helps!

Xn0r
Fanfold Paper
Posts: 9
Registered: ‎07-24-2014
Location: US
Views: 1,827
Message 8 of 9

Re: Stuck with buggy micro code after recent BIOS update

My work T440p started BSODing after the BIOS update and patch too (I believe, was on vacation when it installed).

I also noticed the "WHEA parity errors" in the event log.  I've disabled just the 5175 patch and they went away.

But I'm not sure if that patch is what was causing the BSODs or not.  Time will tell.

Guru
Posts: 612
Registered: ‎12-30-2017
Location: PL
Views: 1,776
Message 9 of 9

Re: Stuck with buggy micro code after recent BIOS update

There is a new microcode update for T440p.

https://support.lenovo.com/us/en/downloads/ds037575

 

I have checked it and accoriding to InSpectre app, it does not have Spectre hotfixes. It seems, this is a package restores previous - stable, CPU microcode.

------
...

Check out current deals!


Shop current deals

Top Kudoed Authors