The tooltip that pops up when downvoting a post reads:
Please consider adding a comment if you think this post can be improved.
Not every post can be improved. Some posts are simply bad and deserve downvotes. Others are simply inaccurate, or orthogonal and do not answer the question properly as written, and deserve downvotes. In these cases, the only way to 'improve' the post is to delete it. A comment would not help to improve it. Beyond that, there is an asymmetry of effort and cost involved; Downvoting requires far less effort than commenting, and does not expose the voter to retaliatory downvoting, or the possibility of getting into a protracted argument.
Beyond that, there are other reasons to downvote as well, quoting Shog9 in one of the meta links linked above:
My personal feeling is that votes - up or down - have their primary value as hints to other readers: which questions are worth reading, which answers should i read first, next, at all. And i vote accordingly, doing what the site lets me to adjust the scores of answers to a given question to where they match my own personal opinion of what matters. I won't vote a decent answer to where it scores < 0, but i see no reason to leave it > 1 if a better answer scores lower, because if i were searching out an answer to the question, i'd want to read the most comprehensive answers first. I don't have a hard and fast rule on this though; sometimes i'll vote on every answer to a question, other times i'll only vote for one or two.
Put another way, why do I need to help you improve your sub-par answer if a better one is already there from somebody else? And if yours outscores theirs, well, I'm going to use my votes to improve the sorting so that the next person who finds this question is more likely to see the better answer first.
But above all else, remember not to get too hung up on downvotes; It only takes 1 upvote to overcome 5 of them, and the down-voter pays a price for each one. They're more important as a sorting mechanism for a site than as a way to hurt your reputation.