I've tried to group todos in to three rough priority categories. Bug Fixes, Quick Improvements and Longer Term Projects. Hopefully those categories are self-explanatory.
Admins/moderators, feel free to add things you've volunteered to do or things that have been broadly agreed upon as needing to be done. I (Fwiffo) haven't gotten around to adding everything yet. Or if you're looking for something to do with the site, but don't have an idea, feel free to take ownership of one of these and get cracking. If you've taken care of one of these, please remove it from the list!