I need to upgrade the RAM in my D20 (4158), with 2 x X5680 Westmere EP Xeons and currently 12 x 8GB RDIMMS installed. The X5680 officially supports up to 288 GB RAM, so could I safely install 12 x 16GB modules? Would it also be possible to mix modules, i.e. 3x16 GB + 3x8 GB per CPU?
12 x 16GB RDIMMs will be fine. Note that mixing 16GB and 8GB modules will work best if you install the 16GB in slots 1/2/3 and 4/5/6 for CPU 1 and 2 respectively, then fill the remaining slots with the 8GB modules.