Displaced Drude peak and bad metal from the interaction with slow fluctuations