The whole point is having the state record the marriage. It's not a permission slip. The officiant of the marriage signs it with witnesses so that everybody agrees that the Rev X married these two people on such-and-such-a-date, these same two people whose identities were recorded by the state earlier for its own records. The officiant agrees, the people involved agree, and somebody else saw it who knows the identities of both people. You get the license in advance so that everybody is clear that it's really these two people, it's not something being filled out after the fact or whatever. It's done this way (rather than, say, having getting the license be replaced by immediately having the state recognize you as married and then having the church perform the marriage, which is the European method, or rather than the reverse) because the churches want to insist that they're not married until the actual wedding and they want to insist that we somehow guarantee that when we perform the church marriage that they actually do end up being married. You want to prevent scoundrels from getting "married" but then never bothering to file the paperwork and then absconding.
Atheists need the license, of course, because they're registering their marriage with the state. How they have their marriage officiated, however, is their own concern. Some get a judge to marry them like immediately after and only have a reception, some find somebody else authorized to perform marriages...