Yes, you do have to know what you are talking about before you vote to close.
The only people who have the global power to vote to close on the network are community managers and developers. They're like the exception handlers' exception handlers: they act when shit really does hit the fan. Their power is extraordinary as they can close questions without understanding what the question is talking about — because you don't do that.
Why do we close questions to begin with? We are very picky with our questions because we are very picky with our answers: we want answers, not blog posts or opinion pieces or "me too"s or "thanks!" or follow-up questions or books on any given topic. Yet surely you don't need to know anything about videogames when a question asks "What is the best engineer hat in Team Fortress 2", right? I mean, hats are purely cosmetic, right? They don't affect the way you play, right?
Oh, look. Maybe you do need to know that a non-unusual Hotrod, in fact, does give you an objective strategic advantage in the game to handle this question no other hat in the game gives you. Maybe there is a "best" hat, after all? Maybe you do need to know what you're talking about before you close questions?
Even when you do discount these bugs, there might still be a right answer might be "any hat so long as it isn't the Gibus". (For an engineer the "I have a hat, you can trust me" effect probably isn't very important, though; you can make a number of arguments here, but if you haven't played TF2 you can't.)
Of course the main problem with this kind of question is that, when the answer is a negative, proving that it is so becomes much harder than asking the question. Do you know all of the bugs and quicks of TF2? I sure don't. "No you can't do that"/"No you can't know that" answers really don't work: "mods can do anything" one might say; "you just didn't look hard enough" another might retort, even if the correct answer at the time of posting actually was negative.
What's the difference between voting to close a question because you don't think there is an answer and answering that you don't think there is an answer?
Four people — plus all those that come after and don't reopen the question.
If five people think there is no non-speculative answer to a plot question (and there aren't five more who reopen it), it's fair to say that the community thinks there is no answer we would allow. This means that the community thinks it is futile to let the question be open for answers we would downvote and delete anyway for being helplessly speculative.
If those five people happen to be wrong, someone who knows better can comment so, use his vote to reopen and/or ask others to do so.
This is different from having an answer with four upvotes. Upvoting answers can become instinctive, second nature: "yeah this looks right." More people have the right to upvote than they have to downvote, too. On the other hand, bar for closing and reopening is the same, however. It's also a high bar: we ask people to know what they're talking about and understand how this site rolls before they can make this kind of call. If five highly-experienced users can reach the same conclusion without it being overturned, I feel that's more authoritative than having an answer with four upvotes.
Additionally, voting is anonymous; answering isn't. You might very well know for certain that there isn't an answer to a question, and yet you don't want to engage the asker and get into an argument about it. Why should the site ask any one of us to be the one person who steps up to say the obvious and gets all of the inevitable notifications for it?
As a moderator, I sure do occasionally miss the opportunity to not be the one person who makes the right call. We abstain from opening or closing questions frequently in order to not be that one guy. It's not a pretty place to be, and most people who commented about downvotes should know how that can turn out.
"But badp, what if we don't have five people out of 254 3k'ers who do think there can't be an answer?"
Oh dear, what then? What happens then is that, huh, the question doesn't get closed. So long as we do delete subsequent speculative answers, the site is no worse for having a question open and unanswered. It's way better, in fact, than closing an answerable question.
"But badp, doesn't that mean that it'll leave us with a lot of unclosed lore questions? Surely we're about to drown in a sea of poorly-written fan fiction!"
You don't know that. Is there a history of lore questions getting speculative answers? Do we expend a lot of effort downvoting and deleting them? I don't know. I don't think so. Although, if you will, I'm just one person; maybe if four more agreed with me... :D