Considering we've both been married and his marriage resulted in two kids, yeah, we have to face that the other had previous relationships pretty often, and we do things together that we did with former spouses.
I think what makes clear that we're past a point where we'd have feelings about such things is that we can talk those things with the same level of casualness that we would anything else. For example, when we were planning a Disney vacation, I asked if he'd eaten somewhere before. He said he had, with his ex-wife and her family. I asked if he wanted to eat there again, he said it didn't matter, the food was neither great nor terrible, and while he could eat there again, he thought that there are places that have better food. That was the end of it.
If he said "I ate there with my ex wife on our honeymoon, she had the chicken and I had the beef and I remember how we laughed over how fun the waiter was and enjoyed how romantic the setting was" or even "I ate there with my ex, so I'd prefer to eat there again. Going there reminds me of her and our honeymoon" then there would be a red flag for me. Both tip off that there might be feelings or raw emotion still there.
Really, even when there is overlap on our experiences and an experience we had an ex, it doesn't matter. It's like something that happened in another life. It's neither good or bad, it just is.