From Aquinas' commentary on the 10 commandments:
"It must be known that the Commandment, “You shall not commit adultery,” not only forbids adultery but also every form of immodesty and impurity. There are some who say that intercourse between married persons is not devoid of sin. But this is heretical, for the Apostle says: “Let marriage be honorable in all and the bed undefiled” [Hb 13:4]. Not only is it devoid of sin, but for those in the state of grace it is meritorious for eternal life. Sometimes, however, it may be a venial sin, sometimes a mortal sin. When it is had with the intention of bringing forth offspring, it is an act of virtue. When it is had with the intent of rendering mutual comfort, it is an act of justice. When it is a cause of exciting lust, although within the limits of marriage, it is a venial sin; and when it goes beyond these limits, so as to intend intercourse with another if possible, it would be a mortal sin.
Adultery and fornication are forbidden for a number of reasons. First of all, because they destroy the soul: “He who is an adulterer has no sense, for the folly of his heart shall destroy his own soul” [Prov 6:32]. It says: “for the folly of his heart,” which is whenever the flesh dominates the spirit. Secondly, they deprive one of life; for one guilty of such should die according to the Law, as we read in Leviticus (20:10) and Deuteronomy (22:22). Sometimes the guilty one is not punished now bodily, which is to his disadvantage since punishment of the body may be borne with patience and is conducive to the remission of sins; but nevertheless he shall be punished in the future life. Thirdly, these sins consume his substance, just as happened to the prodigal son in that “he wasted his substance living riotiously” [Lk 15:13]. “Do not give your soul to harlots, lest you destroy your inheritance” [Sir 9:6]. Fourthly, they defile the offspring: “The children of adulterers shall not come to perfection, and the seed of the unlawful bed shall be rooted out. And if they live long they shall be nothing regarded, and their last old age shall be without honor” [Wis 3:16-17]. And again: “Otherwise your children should be unclean; but now they are holy” [1 Cor 7:14]. Thus, they are never honored in the Church, but if they are clerics their dishonor may go without shame. Fifthly, these sins take away one’s honor, and this especially is applicable to women: “Every woman who is a harlot shall be trodden upon as dung in the way” [Sir 9:10]. And of the husband it is said: “He gathers to himself shame and dishonor, and his reproach shall not be blotted out” [Prov 6:33].
St. Gregory says that sins of the flesh are more shameful and less blameful than those of the spirit, and the reason is because they are common to the beasts: “Man when he was in honor did not understand; and became like senseless beasts that perish” [Ps 48:21]."
From the Summa:
"I answer that, In marriage there is a contract whereby one is bound to pay the other the marital debt: wherefore just as in other contracts, the bond is unfitting if a person bind himself to what he cannot give or do, so the marriage contract is unfitting, if it be made by one who cannot pay the marital debt."