Nothing prevents one from going home to one’s family except prayer.
Sunan Abi Dawud Book 2, Hadith 80