The Messenger of Allah (ﷺ) said: "If you say to your companion: 'Be quiet and listen' when the imam is delivering the Khutbah, you have engaged in idle speech."
Sunan an-Nasa'i Book 19, Hadith 22