Well, to be fair, the framers were divided on the issue.
People will often cite quotes like the on you have above to make a claim that it was intended to be a religious nation (and we can get into the problems and pitfalls of that if you'd like) ...or the slight variation of that same sentiment which is what you expressed, which is "this limited government stuff only works if everyone believes in God"
But then neglect to mention that other framers/founders made statements like these (all made by different framers)
The Bible is a history of wickedness that has served to corrupt and brutalise mankind.
The United States of America should have a foundation free from the influence of clergy.
Christianity neither is, nor ever was a part of the common law.
The purpose of separation of church and state is to keep forever from these shores the ceaseless strife that has soaked the soil of Europe with blood for centuries.
In every country and in every age, the priest has been hostile to liberty. He is always in alliance with the despot, abetting his abuses in return for protection to his own.
Lighthouses are more helpful than churches.
It's important to note that what's considered an atheist or agnostic today, would've often been referred to as a "deist" in those times. And a substantial portion of the the gentlemen who helped set up our country fit that mold.
For every John Adams and John Jay, there was a Ben Franklin and Thomas Paine.