I did a quick search of
http://www.dpreview.com and my first recommendation would be the Nikon Coolpix 4300. You can probably get it for $250 which would leave you with enough money to get a good memory card, a tripod, Photoshop Elements etc...
The slightly more expensive cameras really don't add much to the package, so I don't think it would make much sense to spend your money on the little extras. For example, it only has 3 ISO settings, no audio on the video, and does not support type 2 compact flash. You could spend an extra hundred dollars to bump up all the little features, but the truth is you won't miss them.
What I looked for (when I did the features search on the site) was the manual settings. To really learn photography well (as you seem to want to) you need to be able to play with aperture and shutter priority, white balance, and manual focus. To be VERY honest, you do not need these features to take good pictures, and it's likely that you won't often use them. However, for the price range you're looking at, it's not hard to find a good camera with those features, and it gives you more flexability to compose your shots as you become more proficient at photography.
Anyway, what really sells me for this camera is it's low price along with 4 megapixels, 3x zoom, a Lithium-Ion battery (rather than the AAs), and compact flash cards. You really won't need more megapixels than that, you won't be able to get more zoom unless you pay a lot more or sacrifice overall quality, the AAs are just annoying in my opinion (and LI batteries have few downsides), and compact flash cards are generally the best right now, though when you upgrade to another model in a few years, you'll likely to get an entirely different format.
Check out the reviews and articles online about what is important and what's not. I especially like
http://www.dpreview.com because it gives such an easy comparison of each camera's features. You should notice that the prices are often a bit dated, and many of the cameras don't have in depth reviews, so it's important to go to other sites to look for any glaring design flaws in a particular model, or to get different views on what's important in a camera.