Spatial Modulation (SM) is a recently proposed single-RF multiple-input-multiple-output (MIMO) technique, which is capable of outperforming many conventional MIMO transmission schemes with low implementation and computational complexity. Recently, there have been some attempts in understanding the performance of SM in multi-user environments. However, most of the work has been oriented towards uplink multi-access scenarios. Also, conventional downlink/broadcast MIMO precoding techniques such as Zero Forcing (ZF) or Minimum Mean Square Error (MMSE) cannot be used in Multi-User SM (MU-SM), as part of the data in SM is also encoded into the Channel Impulse Responses (CIRs). In this paper, a novel precoding scheme for single-cell downlink MU-SM systems is proposed with a two-fold objective: i) the precoder needs to be able to completely eliminate the Multi-User Interference (MUI) by taking advantage of the Channel State Information (CSI) at the transmitter and ii) it needs to allow the users to use a single-user Maximum Likelihood (ML) optimum detector while achieving the same performance as interference-free point-to-point SM transmission. Finally, we also develop an interference-aware multi-user detection scheme, which does not require any CSI at the transmitter, and compare its performance with that of single-user detection schemes based on precoding.