Mu’aiqib reported the Prophet (ﷺ) as saying ; Do not remove pebbles while you are praying; if you do it out of sheer necessity, do it only once to smooth the pebbles.
Sunan Abi Dawud Book 2, Hadith 557