I heard AbulQasim (ﷺ) who spoke the truth and whose word was verified say: Mercy is taken away only from him who is miserable.

Sunan Abi Dawud Book 43, Hadith 170