When a man calls his brother an unbeliever, it returns (at least) to one of them.

Sahih Muslim Book 1, Hadith 121