So I fixed my RSS and Atom feeds on this Wordpress 2.5 powered blog. Apparently there was an extra blank line in one of my theme function files. This put a blank line at the top of the XML documents making them invalid.
Interestingly, it also fixed the issue I was having with the Visual editor not working for new posts. I have been beating my head against the wall for a few days on that one. All better now!