wow - really interesting idea. really troubling actually. depends why it is that you hate them, I suppose. I think it would have to be a very specific kind of hate, wouldn't it? For instance, if you loved their work then surely we would have to deduce that you respected them or esteemed them quite highly on some level to begin with?
So if we're dealing with an emotion that combines hate and respect I guess we have to ask ourselves what kind of experience of spectatorship that could arise from. I think the main cause for this kind of dynamic is when you get to know the artist, they blow you off for whatever reason, you admired them before and then you're left with this bittersweet aftertaste that stays with you. For masochists, the exquisite pain that revisits them every time they look at their work could render the art even more alluring and exciting.
I guess it's touching on the idea of reception and experience or perhaps the issue of spectatorship being personalised by the emotion of direct experience.