Usually when I actually say "I hate you" aloud to another person, it's to my wife, but that's probably more about statistical likelihood, given that I say probably a good 70-90% of all words on a given day to my wife. I've probably used the phrase endearingly before, and I'm certain my wife has used it for me. I can't remember a single time I've said it to someone else, though, except maybe my sister when we used to fight. (Would guess that the people you say "I hate you" -- and mean it -- would almost have to be a close relation, probably family.)

But most of my use of "I hate you" is more likely expressing hostility toward inanimate objects. Computers, mostly. "LOAD! I *HATE* YOU, *LOAD*." I used to have some anger management issues that were expressed almost entirely at my videogames, which I would throw around, scream at, curse at, otherwise do terrible things to. I've stopped the screaming for the most part, but I still say "I hate you" to objects a lot, I think (when I use the phrase -- most often I "hate" ideas or situations, not people or things; "I hate it when..." or "I hate how...")