Version controlling my home dir

For awhile now I've noticed that things in my home dir aren't setup optimally for my work flow. I've been trying to run with SELinux enabled and I run into development problems when I try to run web applications out of my home directory. Various parts of my home directory are version controlled separately due to the software projects they're part of, but not as a whole.

What I'd like is to setup some other place for "projects" (Mozdev code and docs, other software projects, RPM building, etc) and then version control my home dir.

The problem is I'm not sure how much this helps me. Things like IM clients still are going to want to log things to \~/.somedir/log which is evil. SELinux contexts for files in /home/myprojects is still going to be wrong; I'm not going to be able to run webapps out of there, either. Moving my docs out of my home dir might be a pain due to xdg-user-dirs needing to be setup correctly to point at the document dir

I'd really love to have a lean, mean homedir that is version controlled that I can port around between boxes. Have people attempted this before? What about the above problems?