03-20-2013 07:33 AM
We spent a couple of days cornering what appears to be an issue with the Intel 520 SSD drives in Lenovo x230 laptops. It was first showing up on a clean Debian installation, while installing a guest operating system into a VM. Looking around on forums, there appears to be some people having issues with database workloads too. So I decided to create a small user-space program to repoduce the problem. IMPORTANT: Before you try it, be ready for a system crash requiring a cold reboot.. It's available at:
direct link to .c file:
This program simply performs random-access-writes of 4Kb into a single file.
Executive summary of our findings (the details are in the test-ssd-write.c header in the git repo):
- We reproduced this issue on 4 x230 machines (all our x230 have 180GB Intel drives, and they are all affected),
- We took a SSD from one of the machines, moved it into an x200, and the problem still occurs,
- The problem seems to occur independently of the filesystem (reproduced on ext3 and ext4),
- Problem reproduced by test-ssd-write.c (git tree above): After less than 5 minutes of the heavy write workload, we get SATA errors and we need to cold reboot the machine to access the drive again. Example usage (don't forget to prepare for a computer freeze):
./test-ssd-write somefileondisk 209715200 1234 -z
(see options by just running ./test-ssd-write)
The problem occurs with drive model SSDSC2BW180A3L, with both firmwares LE1i and LF1i (those are Lenovo firmwares). We could reproduce the issue on 3.2 (Debian), 3.5 (Debian), 3.7.9 (Arch) distribution kernels. We could reproduce it with x230 BIOS G2ET90WW (2.50) 2012-20-12 and G2ET86WW (2.06) 2012-11-13, but since it can be reproduced on a x200 too, it does not appear to be a BIOS issue.
We tried the program on a range of other SSD drives, one of those including the same SandForce 2281 controller (details within test-ssd-write.c header). So our current guess is that the Lenovo firmware on the SSD might be part of the problem, but it might be good if we could to confirm that Intel's firmwares work fine.
Thoughts, ideas, hints about who to contact on this issue would be very much welcome,
03-20-2013 08:25 AM
I can confirm the exact same problem with Debian and Ubuntu.
I can reproduce it easily on Debian by installing a VM.
On Ubuntu it is harder to reproduce but I already hit this problem when installing updates (dpkg does a lot of sync/fdatasync just like your test program).
I think the difference between the 2 distros that explain why it is easier to reproduce on one is the default I/O scheduler (cfq for Debian and deadline for Ubuntu).
Lenovo already completely replaced my laptop (X230 with SSDSC2BW180A3L) and I still have this problem.
We need a firmware fix.
03-20-2013 12:40 PM
is there maybe a firmware update available for this SSD via intel? I don't expect Lenovo would update the firmware of the drive before installing.
03-20-2013 01:05 PM
The firmware on the x230 Intel SSD drive is made by Lenovo. The last version available (which I tried) is LF1i, published within their SSD firmware update utility FWSS28 on February 14, 2013.
Indeed, trying the test-case with an Intel firmware might help pinpointing the issue, but I did not deploy non-Lenovo firmware on my SSD drive, fearing it might void my warranty.
03-20-2013 01:23 PM
I was thinking the same thing after I wrote that.
I would say as a last result you could probably try the intel firmware, but I believe you are correct in regards to the warranty. Worst case, unless you brick the SSD, I don't see why you couldn't swap intel and Lenovo firmware, it might run faster! lol
Sorry I can't be of more help Mathieu.
03-21-2013 11:10 AM
FWIW, I've been using my X230 with Intel OEM SSDs and haven't seen a problem with them.
I also have a bunch of SandForce based SSDs from Intel and OCZ that I use in various machine and that don't reproduce the behaviour that Mathieu and Julien showed me, it's therefore quite possible that it's the Lenovo-specific firmware that's the cause of those problems.
03-22-2013 08:19 AM
I got the same errors on my own Lenovo X230 (2306-CTO), that I received with an Intel Series 520 180G SSD, also with firmware LE1i. This forced me to replace the drive -- the errors cause users to lose trust in the safety of their data.
03-22-2013 08:30 AM