I think this is his fault. I blame him more than you.

First, he's the one who broke the commitment he had made, by repeatedly cheating on his wife. You had made no such commitment. Blaming the affair partner more than the cheater, is backwards because one owes much more to one's life partner than they do to some stranger.

Second, he treated you as the side chick, available at his convenience rather than according to your needs, and took no responsibility whatsoever for his own child. Even from a legal perspective, he needs to at least pay child support. This is yet another reason cheating on your partner is bad. It creates conflicting loyalties your partner hasn't signed up for. Whatever decision the cheater makes in that type of situation, is wrong.

Congratulations on your motherhood! It seems you wanted this child badly.