Nah, that part's fine. I think the part where you basically categorize anyone that complains too loudly or too negatively as a whiner that annoys you, is probably where you start to get less popular.
It's one thing to say that you're fine with how things are. The Devs should hear from those who are not annoyed overly by the bugs and weird mistakes that sometimes occur. They need positive reinforcement too, they're only human after all. But if you feel the need to tell people to "shut up" basically, because their overly negative complaints "annoy" you, I'm sure some people just feel like telling you to "shut up" in turn. And the rep system is a passive-aggressive way of doing so without getting into a heated debate.
Personally, I prefer heated debates. But I can see why others would just hit neg. I'm sure I'm guilty of it too, when I feel someone is being insulting. Sometimes I actually regret it when my anger gets the best of me. I wish there was a "retraction" button, so that someone could take it back once they calm down a bit and realize that it wasn't really as insulting as it first sounded. I guess a person could report their own neg repping and ask the Devs to take it away from the person they gave it to?
The worst thing is when I've accidentally negged someone I actually wanted to pos rep. I've always sent them a PM apologizing and so far everyone's just laughed it off. Which is easy to do since the rep system is really meaningless. I'm not even sure if too much negrep will put someone in moderation like it was initially supposed to. So it's just a gauge I guess of if people disagree and to what degree they do.