The Prophet (ﷺ) used to pray (the Nawafil) on his Mount facing east and whenever he wanted to offer the compulsory prayer, he used to dismount and face the Qibla.
Sahih al-Bukhari Book 18, Hadith 19
← Previous Hadith At night `Abdullah bin `Umar used to offer the prayer on the back of his animal during the journey and never cared about the direction he faced. Ibn `Umar said, "Allah's Messenger (ﷺ) used to offer the optional prayer on the back of his Mount facing any direction and also used to pray the witr on it but never offered the compulsory prayer on it."

Sahih al-Bukhari Book 18, Hadith 18 Next Hadith → We went to receive Anas bin Malik when he returned from Sham and met him at a place called 'Ainat- Tamr. I saw him praying riding the donkey, with his face to this direction, i.e. to the left of the Qibla. I said to him, "I have seen you offering the prayer in a direction other than that of the Qibla." He replied, "If I had not seen Allah's Messenger (ﷺ) doing it, I would not have done it."

Sahih al-Bukhari Book 18, Hadith 20