Well
that is probably a good question.
When you see something like a piece of "fossilized wood" like say this one:
There's still visible RINGS in there. But there's no more wood. So what happens when wood turns into THIS stuff.
This is mostly silica in some cases (this looks like some from the Petrified Forest in AZ so I think it's SiO2.)
Now we have to get the wood:
1. Preserved without oxidizing away (if you cut down a tree in your yard and let it set out there it will soon rot and oxidize away to nothing)
2. Bury it beneath some sediment and let it get exposed to groundwater which is carrying SiO2. SiO2 doesn't dissolve in groundwater very much. It depends on pH and can get much more soluble in high pH regions, but in regular water the solubility is only about 80mg per Liter (rough estimate from Wikipedia).
3. You need to move groundwater through the rock carrying tiny amounts of SiO2 and fill the spaces in the wood with SiO2 until you preserve the structure of the original tree. Down to the rings.
Now, not all fossils are like that. In some cases some fossils aren't "permineralized"(like the tree example above) or if they are they are replaced by more soluble materials like CaCO3 (calcium carbonate).
But in general it takes some time. But not all fossils are "replaced". Some things like bivalves and shells can remain as they are or they can even be dissolved and replaced by other minerals.
How people know this takes time is understanding a few key factors:
1. Getting the bones or shells or plant materials into a position where they can be preserved
2. If replacement or permineralization occurs you have to know how quickly water moves through the sediment/rock around it and the
solubility of the replacing phase. Water moves somewhat slowly through sediment and when you couple that with low solubility materials you come up with a lot of time needed.
Now I'm sure that not all fossils take equally long to preserve and create, the ones that are like hard shells that don't get replaced take as long to "make" as it would the rock around them in which they are preserved.