If a question is unclear and it can't be answered in its current state, voting to close it as "Unclear what you're asking" is perfectly fine, I'd even say it's expected.
Should the question then be edited before accumulating 5 close votes, and should such edited question be off-topic because it becomes clear that it's asking for tech support with modded Minecraft, that question could then receive the rest of its close votes under that specific reason, with the end result of it still getting closed. Keep in mind that "modded Minecraft tech support" is a custome close reason, so the asker will be notified of it even if only one person votes to close for that reason.
Should the question be edited after its fifth close vote, well... that's what the Reopen Votes queue is for. If you notice a question that was closed as "unclear", but needs to remain closed after clarification, leave a comment that explains why the question is not getting reopened, and click the "Leave Closed" button.