I too had this problem but I think I may have a solution. All I needed to do was delete the offending podcast's folder from Windows explorer (where the Zune software keeps the podcast files). This presented a problem since the folder was stuck on read-only and couldn't be deleted. I had to start Windows in Safe Mode by pressing F8 at start up. I then found the folder and deleted it, and it has stayed deleted since. You might have to access the properties of the folder in safe mode and give yourself the correct permissions from the securities tab. I'm running XP so Vista or Win7 might not have the same Safe Mode option that I did, but I hope this works for everyone else. Good luck!
It's Saturday night. I have no date, a 2 liter bottle of Shasta, and my all Rush Mix Tape... Let's rock!