cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
mikewinddale
Ctrl-Alt-Del
Posts: 16
Registered: ‎12-21-2016
Location: US
Views: 14,596
Message 1 of 29

E570: needs better NVMe driver support

So I recently bought a new Lenovo E570, which includes a Toshiba XG4 model M.2 NVMe SSD. The Toshiba XG4 has model number "thnsf5256gpuk".
 
Long story short, without proper drivers, the performance is absymal. Absolutely horrifically awfully terrible. Lenovo should take note of this issue. Luckily, I was able to jury-rig a workaround, so let me tell the full story:
 
Because the XG4 is an OEM drive (i.e., it is sold only to computer manufacturers, not retail to consumers), it is difficult to find information about it, but it does appear in Toshiba's catalog: https://toshiba.semicon-storage.com/info/docget.jsp?did=55015
 
In my Lenovo E570, I added a retail Kingston KC400 SATA SSD as personal storage. So now the computer has two SSDs: an OEM Toshiva M.2 NVMe as the Windows boot drive, and a retail Kingston SATA SSD as personal storage.
 
I ran some benchmarks to compare the two SSDs, and I found that the Toshiba had extremely subpar 4K random write performance. Basically, what this means is that if you try to write a lot of very small files (instead of a few large files), your Toshiba SSD will struggle and crawl. The Toshiba performed decently in 4K read, and it trounced the Kingston in sequential tasks (because it is NVMe), but in 4K write, the Toshiba was absolutely abysmal.
 
Specifically: whereas the Kingston KC400 got about 70 to 90 MB/s in 4K write, the Toshiba was getting about 1.5 to 1.6 MB/s. I benchmarked the Hitachi mechanical HDD in my old Lenovo E545, and it got about 0.5 to 0.75 MB/s in 4K write. So the Toshiba was still 3 times faster than a mechanical drive, but it was about 60 times slower than the Kingston SSD.
 
Needless to say, this is a serious problem.
 
I went to look up benchmarks for the XG4 to see if my results were abnormal. Unfortunately, I couldn't find anything about the XG4 online (since it's an OEM drive), but I did see that the Toshiba XG3 (also NVMe) was rebranded as the OCZ RD400. So I looked up benchmarks for the OCZ drive, and it performed comparably to the top-of-the-line Samsung 950/960 Pro in 4K random writes!!! So why was my XG4 performing so abysmally?
 
The next thing I tried to do was find updated drivers or firmware for the Toshiba. Again, no dice. As an OEM drive, there was nothing. Then I updated my BIOS, chipset drivers, Intel RST, etc. (Of course, the Intel RST is for AHCI, not NVMe, but I figured, you never know! Updated drivers rarely hurt!) Nothing helped. 
 
Then I Googled around for anyone else having trouble with 4K writes, and a common denominator seemed to be people disabling write caching. Once they enabled write caching, their performance improved. I checked Device Manager, and write caching was already enabled for my Toshiba. But out of curiosity, I disabled write caching on the Kingston, and its 4K writes dropped down to the same 1.5-1.6 MB/s. So at least I knew that my Toshiba was performing *as if* write caching was disabled, even though it was enabled in the Device Manager. Still no solution to the problem, but at least, closer to a diagnosis.
 
So I started reading anything I could find about SSDs and write caching. I couldn't find much. But finally, I came across several extremely helpful articles:
 
 
Basically, what these four articles found was that the Microsoft NVMe drivers effectively disabled write caching by using FUA (forced unit access). So this means that if you use the Microsoft NVMe drivers, you won't be able to use the cache that built into your SSD. So whereas the Intel RST drivers successfully implement write caching for the SATA drives, the Microsoft NVMe drivers fail to implement write caching for NVMe drives. According to the articles I just posted, several NVMe drives (including Samsung's) performed poorly until Samsung released their own proprietary NVMe drivers to replace Microsoft's NVMe drivers. In the meantime, people had been having to disable write cache flushing to get decent performance out of the Samsung NVMe drives. (Disabling write cache flushing is a VERY bad idea because it creates a serious risk of data corruption in the event of a power failure or blue screen.) SATA drives were performing fine, but NVMe drives were crippled by the Microsoft drivers. 
 
I called Toshiba and Microsoft and asked them both if they had any plans to release new NVMe drivers anytime soon. Both of them said they had no such plans.
 
So this put me in a tight bind. What was I supposed to do if Toshiba and Microsoft weren't releasing any new drivers?
 
 
Well, I started Googling to find out what would happen if I used another manufacturer's NVMe drivers. Several people posted online that they had successfully used Intel NVMe drivers for non-Intel SSDs. And I saw that even though Samsung has separate firmwares for every drive, all of their NVMe drives use a common Windows NVMe driver. So it appeared to be the case that while every drive needed its own unique firmware to manage the SSD on the lowest hardware level, the Windows NVMe drivers were high enough level to be more or less interchangeable. So I figured, hey, let me try this. Since I'm not touching the firmware, worst case, I just go into safe mode and roll back my drivers to the original Microsoft drivers. As long as I don't touch the firmware, I'm unlikely to damage the hardware. So I backed up all my data and started installing other manufacturer's NVMe drivers.
 
The first one I tried was OCZ's RD400 NVMe driver, since the OCZ RD400 is a re-branding of the Toshiba XG3 (my Lenovo has the Toshiba XG4). I installed it via the EXE wizard installer, rebooted my computer, and ran some benchmarks. No improvement. So I went into Device Manager and checked under "Storage Controllers". I saw that my system was still using Microsoft's "stornvme.sys". Apparently, OCZ's EXE wizard installer hadn't done the job. So I manually told Device Manager to update the driver for my NVMe storage controller, and I told it to show me drivers that are NOT compatible with my device. I looked under "Toshiba" and chose the XG3 driver. I installed it and rebooted my computer. I ran benchmarks. No improvement. So I went back into Device Manager and this time, I chose the OCZ RD400 driver under "Toshiba". I installed it and rebooted. Now, Device Manager showed it was using "ocznvme.sys" instead of Microsoft's "stornvme.sys". When I ran the benchmarks, I saw an ENORMOUS improvement. ABSOLUTELY GIGANTIC.
 
Let me show you all some benchmarks, and you'll see what I mean. So I think this is a valuable story about the importance of replacing Microsoft's NVMe drivers. Hopefully, in the future, Lenovo will choose a brand of NVMe SSD that comes with manufacturer driver support.
 

 

The benchmarks are in this order:

(1) Anvil, with Microsoft's stornvme.sys

(2) Anvil, with OCZ's ocznvme.sys

(3) AS, with Microsoft's stornvme.sys

(4) AS, with OCZ's ocznvme.sys

 

Again, pay special attention to 4K random write:

 

Anvil - Toshiba (Lenovo E570, NVMe).pngAnvil, using Microsoft's stornvme.sysAnvil - Toshiba (Lenovo E570, NVMe) - using OCZ RD400 NVMe driver.pngAnvil, using OCZ's ocznvme.sysas-ssd-bench THNSF5256GPUK TOSHIBA (Lenovo E570, NVMe).pngAS, using Microsoft's stornvme.sysas-ssd-bench THNSF5256GPUK TOSHIBA (Lenovo E570, NVMe) - using OCZ RD400 NVMe driver.pngAS, using OCZ's ocznvme.sys

 

 

Jachin
What's DOS?
Posts: 1
Registered: ‎12-28-2016
Location: US
Views: 14,235
Message 2 of 29

Re: E570: needs better NVMe driver support

Thank you for taking the time to write this up! Very imformative and helpful. I'll try it when I recieve my laptop.

Tetio
Paper Tape
Posts: 5
Registered: ‎12-28-2016
Location: US
Views: 14,170
Message 3 of 29

Re: E570: needs better NVMe driver support

Confirm same tests.. via AS SSD Benchmark

 

((write)) seq: 105.12   / 4k: 1.80    / 4k-64: 2.04     / Acc: 2.198

((read))  seq: 1335.33 / 4k: 28.82 / 4k-64: 108.47 / Acc: 0.058

Score (write): 271 (read): 14 (total) 374

 

This test took ages to run..

mikewinddale
Ctrl-Alt-Del
Posts: 16
Registered: ‎12-21-2016
Location: US
Views: 14,154
Message 4 of 29

Re: E570: needs better NVMe driver support

Glad you could confirm my results! Thanks!

 

I'd be interested to see what happens if you use the OCZ RD400 driver. Again, I had to manually change my storage driver in the control panel, because the OCZ install wizard (EXE) didn't successfully do it. Of course, make sure you create a Restore Point, and I take no responsibility for any failure, etc. etc.

 

Also, after I installed the OCZ NVMe driver, I discovered that now, the OCZ SSD Utility works too! So I've been able to do things like alter my drive's over-provisioning, etc. By default, the drive had no over-provisioning!

 

Some sample screenshots:

 

Toshiba with OCZ utility.png

 

 

mikewinddale
Ctrl-Alt-Del
Posts: 16
Registered: ‎12-21-2016
Location: US
Views: 14,152
Message 5 of 29

Re: E570: needs better NVMe driver support

Jachin, glad to help!

And see my reply to Tetio below. After I installed the OCZ NVMe driver, I discovered that the OCZ SSD Utility works too. It correctly autodetects everything about the Toshiba drive.

Tetio
Paper Tape
Posts: 5
Registered: ‎12-28-2016
Location: US
Views: 14,130
Message 6 of 29

Re: E570: needs better NVMe driver support

Well it seemed to help the card in a simular way as your results..  But I also compared it to my 3 year old kingston hyperx 3k 2.5

 

Improved result with the OCZ driver

improvement.PNG

 

This is the HyperX 3k I installed since I had it and wasn't using it

hyperx.PNG

 

 

will give the utlity tool a try in a bit, still setting up a lot of stuff on the book

Tetio
Paper Tape
Posts: 5
Registered: ‎12-28-2016
Location: US
Views: 14,128
Message 7 of 29

Re: E570: needs better NVMe driver support

still think these write speeds are slower then they should be.

 

 

Yeah this has been an amazing fix to the many issues I have been dealing with.. from sluggish installs to worry about my wifi card not working right..

mikewinddale
Ctrl-Alt-Del
Posts: 16
Registered: ‎12-21-2016
Location: US
Views: 14,116
Message 8 of 29

Re: E570: needs better NVMe driver support

Well, the OCZ SSD Utility says that the M.2 is only getting 2 lanes of PCIe. So that might be part of the problem too. (I've circled it in red).

I'm not sure there's anything to do about this. Seems like a hardware limitation. Your Toshiba M.2 SSD is still faster than your HyperX in most areas, so . . . I guess this is life? You can't have everything, I guess?

 

sssd.png

Tetio
Paper Tape
Posts: 5
Registered: ‎12-28-2016
Location: US
Views: 14,109
Message 9 of 29

Re: E570: needs better NVMe driver support

True that, its a ton better then it was out of the box =)

 

Plus we got a great deal anyway.

 

Thanks for your research!

 

 

mikewinddale
Ctrl-Alt-Del
Posts: 16
Registered: ‎12-21-2016
Location: US
Views: 14,107
Message 10 of 29

Re: E570: needs better NVMe driver support

Yeah, I got mine for about $550. Thanks, Black Friday! For $550, I'm not expecting top-of-the-line performance that will let me run a webserver, LOL. My M.2 is still faster than a SATA, even if it's not as fast as the review sites say it's supposed to be. And any kind of SSD is still hundreds of times faster than a mechanical hard drive. So in the end, it's still good.

Top Kudoed Authors