John, you were referring to this passage:
2Sa 12:15 And Nathan departed unto his house. And the LORD struck the child that Uriah's wife bare unto David, and it was very sick.
2Sa 12:16 David therefore besought God for the child; and David fasted, and went in, and lay all night upon the earth.
2Sa 12:17 And the elders of his house arose, [and went] to him, to raise him up from the earth: but he would not, neither did he eat bread with them.
2Sa 12:18 And it came to pass on the seventh day, that the child died. And the servants of David feared to tell him that the child was dead: for they said, Behold, while the child was yet alive, we spake unto him, and he would not hearken unto our voice: how will he then vex himself, if we tell him that the child is dead?
2Sa 12:19 But when David saw that his servants whispered, David perceived that the child was dead: therefore David said unto his servants, Is the child dead? And they said, He is dead.
2Sa 12:20 Then David arose from the earth, and washed, and anointed [himself], and changed his apparel, and came into the house of the LORD, and worshipped: then he came to his own house; and when he required, they set bread before him, and he did eat.
2Sa 12:21 Then said his servants unto him, What thing [is] this that thou hast done? thou didst fast and weep for the child, [while it was] alive; but when the child was dead, thou didst rise and eat bread.
2Sa 12:22 And he said, While the child was yet alive, I fasted and wept: for I said, Who can tell [whether] GOD will be gracious to me, that the child may live?
2Sa 12:23 But now he is dead, wherefore should I fast? can I bring him back again? I shall go to him, but he shall not return to me.
Note what David said at the end: "I shall go to him, but he shall not return to me." This infers that David's child is in heaven, and David will see him eventually. This would support the idea that babies do go to heaven.