Yahya related to me from Malik that he had heard that Abdullah ibn Abbas and Abdullah ibn Umar were asked about a man who had a free woman as a wife and then wanted to marry a slave-girl. They disapproved that he should combine the two of them.
Muwatta Malik Book 28, Hadith 30