The Prophet (ﷺ) forbade men to use saffron.

Sahih al-Bukhari Book 77, Hadith 63