The Prophet (ﷺ) forbade men to use saffron.
Sahih al-Bukhari Book 77, Hadith 63