By the way, yes, I love talking about this stuff

It's one big reason why I'm a graduate student studying cosmology at the moment.
What do you suppose the universe is expanding into?
It's not expanding into anything. The way in which we describe the universe as expanding is that space is being created all the time at some rate. Imagine drawing a bunch of dots on a balloon, then inflating it. As you inflate the balloon, the dots will get further apart. But the material of the balloon isn't expanding into anything: the dots aren't moving into rubber off the balloon's surface. They're just getting further apart because the rubber itself is being expanded. It's the same with the universe.
When you look in the context of General Relativity, there really is no limit to space. It can be stretched, compressed, and bent, all by the action of the mass that inhabits the space. Right now, the space in our universe is being stretched by the matter that inhabits it.
It had an obvious beginning.
Well, no, not that we can be sure of. We know that immediately before our big bang started, cosmic inflation was occurring. But what we don't know is how long cosmic inflation occurred. It only needed to last for an absurdly minuscule fraction of time to explain the entirety of our universe, but it could have been going on for as long as you like before that. How is this possible? Well, during cosmic inflation you would have had exponential expansion, where the scale factor as a function of time would have been:
a(t) = e^(Ht)
...where H is a constant (the Hubble parameter, sometimes referred to as the Hubble constant, though in principle it isn't always constant). In this context, the universe could be inflating for an infinite amount of time, and, going back in time, we would never reach a time when the universe was a single point. So no, we don't know that there was a beginning.
What we do know is that at some point, the nature of at least some small region of the universe changed to give rise to the universe which we can see. What we don't know yet is how that occurred, and, unfortunately, the answer may always be hidden from us. But it's not going to stop us from trying to find it!