In downlink multiuser multiple-input-multiple-output (MIMO) systems, block diagonalization (BD) is a practical linear precoding scheme that completely eliminates interuser interference. However, its sum rate performance is rather poor in practical signal-to-noise ratio (SNR) regions due to the transmit power boost problem. In this paper, we propose a modified BD precoding scheme to improve its low-to-medium SNR performance with the so-called “effective SNR enhancement” technique. The proposed transmit covariance matrices are obtained by first solving a power minimization problem subject to the minimum rate constraint achieved by BD and then properly scaling the solution until the power constraint is active. The power minimization problem is nonconvex in general. We therefore propose an efficient algorithm that approximately solves the problem. Simulation results demonstrate significant sum rate gains over BD and existing minimum mean square error (MMSE)-based precoding schemes.