“The Messenger of Allah (ﷺ) said: ‘O Banu ‘Abd Manaf! Do not prevent anyone from circumambulating this House or praying at any time he wants of the day or night.”
Sunan Ibn Majah Book 5, Hadith 452