that the Messenger of Allah (ﷺ) said: "A man has more right to his seat. If he leaves for some need of his, then he returns, then he has more right to his seat."
Jami` at-Tirmidhi Book 43, Hadith 21