The following is intended for the silent Muslim audience. I am aware that there are few active Muslim members on the site currently.
There has been some confusion regarding Muslim men allowed to have sex with their slaves. Many commentators agree that this is true and has its basis in the Hadith. However, just because a opinion is popular does not make it a correct one! Fornication is fornication and being unfaithful is being unfaithful irrespective of doing it with a free women or doing it with a slave.
Let us take a look at the story of a Prophet who has been sold into slavery.
And he of Egypt who purchased him said unto his wife: Receive him honourably. Perchance he may prove useful to us or we may adopt him as a son. Thus we established Joseph in the land that We might teach him the interpretation of events. And Allah was predominant in His career, but most of mankind know not. Quran 12:21
And she, in whose house he was, asked of him an evil act. She bolted the doors and said: Come! He said: I seek refuge in Allah! Lo! he is my lord, who hath treated me honourably. Lo! wrong-doers never prosper. Quran 12:23
And certainly she made for him, and he would have made for her, were it not that he had seen the manifest evidence of his Lord; thus (it was) that We might turn away from him evil and indecency, surely he was one of Our sincere servants. Quran 12:24
And they both hastened to the door, and she rent his shirt from behind and they met her husband at the door. She said: What is the punishment of him who intends evil to your wife except imprisonment or a painful chastisement? Quran 12:25
She said: This is he on whose account ye blamed me. I asked of him an evil act, but he proved continent, but if he do not my behest he verily shall be imprisoned, and verily shall be of those brought low. Quran 12:32
My Lord! the prison house is dearer to me than that to which they invite me; and if Thou turn not away their device from me, I will yearn towards them and become (one) of the ignorant.
In their histories there is certainly a lesson for men of understanding. It is not a narrative which could be forged, but a verification of what is before it and a distinct explanation of all things and a guide and a mercy to a people who believe. Quran 12:111
The following verses are self explanatory. That is trying to sexually abuse one's slave is an evil act, and being unfaithful to one's spouse be it with a free person or with a slave is perhaps even worse. Allah prohibited a Prophet from doing such stuff, do you think he will allow us to do such stuff? Look how Prophet Joseph is saying that he would rather go to prison than to commit fornication and such indecency.
Quranic verses 4:3, 4:24, 4:25 all speaks of marrying your slaves. Let us demonstrate on 4:24:
Also (prohibited are) women already married, except those whom your right hands possess: Thus hath Allah ordained (Prohibitions) against you: Except for these, all others are lawful, provided ye seek (them in marriage) with gifts from your property,- desiring chastity, not lust, seeing that ye derive benefit from them, give them their dowers (at least) as prescribed; but if, after a dower is prescribed, agree Mutually (to vary it), there is no blame on you, and Allah is All-knowing, All-wise.
Sexual relations with slaves are allowed provided that you are married to them.
The only place where there might be a slight hint of Muslims allowed to have sex with their slaves is mentioned in 23:5-6.
Save from their wives or the (slaves) that their right hands possess, for then they are not blameworthy, Quran 23:6
The following verse is actually best translated by Jew/Agnostic who has later embraced Islam namely Muhammad Asad. Now, I do not usually bring any commentators or hadith to prove my point. It is evident that these verses never talks about allowing sex outside of marriage even with slaves; such notions are contradictory to the basics of Islam.
[not giving way to their desires] with any but their spouses - that is, those whom they rightfully possess [through wedlock]
And here is his commentary:
M Asad wrote:[Lit., “or those whom their right hands possess” (aw ma malakat aymanuhum). Many of the commentators assume unquestioningly that this relates to female slaves, and that the particle aw (“or”) denotes a permissible alternative. This interpretation is, in my opinion, inadmissible inasmuch as it is based on the assumption that sexual intercourse with ones female slave is permitted without marriage: an assumption, which is contradicted by the Quran itself (see 4: 3, 24, 25 and 24: 32, with the corresponding notes). Nor is this the only objection to the above-mentioned interpretation. Since the Quran applies the term ‘‘believers” to men and women alike, and since the term azwaj (“spouses”), too, denotes both the male and the female partners in marriage, there is no reason for attributing to the phrase ma malakat aymanuhum the meaning of “their female slaves’’; and since, on the other hand, it is out of the question that female and male slaves could have been referred to here it is obvious that this phrase does not relate to slaves at all, but has the same meaning as in 4: 24 - namely, “those whom they rightfully possess through wedlock (see note on 4: 24) - with the significant difference that in the present context this expression relates to both husbands and wives, who “rightfully possess” one another by virtue of marriage. On the basis of this interpretation, the particle aw which precedes this clause does not denote an alternative (“or”) but is, rather, in the nature of an explanatory amplification, more or less analogous to the phrase “in other words” or “that is”, thus giving to the whole sentence the meaning, “save with their spouses - that is, those whom they rightfully possess [through wedlock]”, etc. (Cf. a similar construction 25: 62 - ‘‘for him who has the will to take thought -that is [lit., “or”], has the will to be grateful”.)]
Asad explanation is far more superior to the one that I could have given you. Indeed, why did most of the translators translated the term "spouses" to "wives"?
Let us look at another verses 24:32-33:
And marry those among you who are single and those who are fit among your male slaves and your female slaves; if they are needy, Allah will make them free from want out of His grace; and Allah is Ample-giving, Knowing.
Let those who find not the wherewithal for marriage keep themselves chaste, until Allah gives them means out of His grace. And if any of your slaves ask for a deed in writing (to enable them to earn their freedom for a certain sum), give them such a deed if ye know any good in them: yea, give them something yourselves out of the means which Allah has given to you. But force not your maids to prostitution when they desire chastity, in order that ye may make a gain in the goods of this life. But if anyone compels them, yet, after such compulsion, is Allah, Oft-Forgiving, Most Merciful (to them),
Read these verses carefully do you see Allah telling us that if we cannot marry we should remain chaste except those whom we posses through our right hand??? No! Does Allah say that if you cannot find a way to get married then go find yourself a slave?? No! As I have said such notions are contradictory to the basics of Islam.
In the end, sex with slaves are simply fabrications of such men who sought to use religion as a means to spread their personal desires. Sexually abusing one's slave is an evil act which relates to nothing but immodestly and indecency!