We know that babies go to heaven when they die from the Bible. There is a passage in the Old Testament about David losing his infant son that was a result from having an adulterous relationship with Bathsheba.
2 Samuel 12:22-23 NIV: ...While the child was still alive, I fasted and wept. I thought, ‘Who knows? The Lord may be gracious to me and let the child live.’ But now that he is dead, why should I go on fasting? Can I bring him back again?
I will go to him, but he will not return to me.
Nathan Rebukes David - The LORD sent Nathan to David. When he came to him, he said, “There were two men in a certain town, one rich and the other poor. The rich man had a very large number of sheep and cattle, but the poor man had nothing except one little ewe lamb he had bought. He raised it...
www.biblegateway.com