"The Prophet forbade men to perfume themselves with saffron. (Shahih)

Sunan an-Nasa'i Book 24, Hadith 88