I have a Lenovo Thinkcentre M83 Tower with an i7-4770, 8 GB of RAM, 500 GB Western Digital Hard drive. I've tried installing a discrete graphics card (an OEM Geforce 605) and the system won't boot: it either freezes or displays the dreaded error 1962. Please note that the system successfully boots  to Windows 10 if the integrated Intel HD Graphics is used (and the discrete graphics is removed)

What I've tried so far hardware wise:

• clearing CMOS
• installing a Geforce 605 to another PCI-e x16 slot
• installing a Radeon HD 7350
• installing a Quadro 2000
• replacing HDD
• replacing HDD cables

What I've tried so far software / settings wise:

• resetting BIOS / UEFI
• updating BIOS / UEFI to the latest 03 July 2018 release
• setting graphics adapter in UEFI from Auto to PEG (PCI Express Graphics)
• enabling CSM
• disabling CSM
• setting boot priority to UEFI Only
• setting boot priority to Legacy Only
• enabling Secure Boot
• disabling Secure Boot
• setting boot priority to the primary HDD
• setting boot priority to boot from LAN

You verified that the GT 605 is good by fully installing it in a different system? 

I can't explain why the GT 605 not working.  However, you should know that it's performance is half that of the built-in Intel graphics.   HD 4600 has a Passmark score of 646, which is twice that of a GT 605 (score of 319), and close to the score of a GT 630 (734.)   

I'm aware of the performance of the card and I'm aware of the integrated grapchis perfromance, too and the performance is not the point. What bothers me is that this PC isn't booting with any of the three graphics cards I've mentioned in my original post and I'm afraid it won't boot with other cards either. The Geforce 605 (or the HD 7350 or the Quadro 2000) aren't the issue here.

Today I got lucky, found an SFF version of the M83 and installed all three graphic cards in it (separately of coruse) and they do work: the PC boots up, POSTs, Windows loads successfyully and so does Ubuntu. Both M83 (the Tower and the SFF) have the same BIOS version and identical BIOS settings.

My mistake, I understood that you when tried the HD 7350 and Quadro 2000 they did work, you only wrote that the 605 did not work.