Rebecca Chernoff asked: How would you handle a question that gets lots of upvotes/answers that the community likes/supports, but is off-topic?
Tom Wijsman answered: If there is another Stack Exchange community available where this question would be considered on-topic, it shouldn't hurt asking them if they want the question and migrating the question there. If the community has upvoted and answered the question in a way that is not conflictive (subjective answers with discussions in their comments) and does have value in terms of learning or usefulness; then there is no direct need to close the question.
Tom Wijsman continued: Now, if an user would flag this question for being off-topic; I would leave a comment to him or explain him my reasoning in the chat. If he however disagrees or multiple users flag the question for being off-topic, I would leave a question on meta asking where people could vote if the question should be closed and look out for a certain amount of votes over time. There is no hurry, it's an useful question that you learn from in the first place…
Ivo Flipse answered: Burn it with fire
Ivo Flipse continued: Users shouldn't forget that deleting and closing isn't final. If there are convincing arguments for reversing it, nobody will object. Its strange how everyone acts like these things are set in stone.
Arda Xi answered: The same as any question really. I learnt very early in this site's infancy that upvotes are about the quality of the question, not about whether it should stay open or closed. There are very well-written but still off-topic questions, and these are the most harmful because they can be a 'foot in the door' to get more and more off-topic questions through.
Mana ಠдಠ answered: My actions would be dependent on whether or not the question is clearly off-topic or more borderline. For the former (for example, in the case of a game-rec question) I would close the topic first before it gets too out of hand, leaving a comment explaining why it's off-topic. It's important to emphasize that these topics aren't allowed on the site. I would be open to a discussion in chat about why it's off-topic, but in order to reverse the decision and reopen the question...
Mana ಠдಠ continued: ...I would need to be thoroughly persuaded and a meta topic would be required. If we allow even a single one of these questions, it redefines the scope and nature of our site, so that's not my call to make alone.
Kevin Y answered: Stack Overflow has some questions that are only around for "historical significance", because they were highly voted before the rules were set in stone. However, in the case of Gaming, I believe we've made our rules pretty clear already, which will prevent most of these questions from appearing (they would be voted against by the community before they become super popular).
Kevin Y continued: If, though, the community happens to support an extremely off-topic question, I'd voice my opinion on that, and see if the community agrees.
fredley answered: First of all, all the things which I can do anyway - start a meta topic and discuss with the community if/why it's off topic, and try and work out a way of re-phrasing the question. This should hopefully sort out a resolution, but if the question still remains un-closed, and is clearly off-topic (i.e. a game recommendation) then I would close it!
Anna Lear answered: We deal with this a fair bit on Programmers. How I handle these cases depends on the question. If the question is clearly off-topic for the site, I support closing it (especially if it receives flags or a few close votes).
Anna Lear continued: If the question is a bad fit for Stack Exchange but is still ostensibly on topic, I prefer to try and turn it into a better question through editing or even just moderating the answers to encourage a high quality of responses. If the question is in a gray area and has flags, I am likely to close it and open a post on meta to see what the general consensus is. A closed question can always be reopened, so the community always has a say in whether or not it stays closed.
Anna Lear concluded: Last but not least, any action I take on a popular question is accompanied by comments explaining the reasons. I'm not a fan of unilateral unexplained closures.
Juan Manuel asked Anna Lear: I find myself disagreeing (just a little) with some of your flags. How do you plan to resolve this with the other mods if you are elected? How do you do it on programmers?
Anna Lear answered: On Programmers, I tend to wait for confirmation - either multiple flags or close votes. I also check with other moderators a fair bit and leave comments on the question asking for clarification or making a note of potential pitfalls. I intend to continue doing this here. It doesn't take much to ask for a sanity check and everyone's usually better off for it.
FallenAngelEyes answered: It partially depends on the severity of how off-topic the question is. If it's gotten enough visibility that it's getting upvotes despite not being on topic, then it means that a number of users has seen it and/or have an opinion on it. I feel that the best course of action would be to open a Meta topic on the question at hand, and depending on the severity of its off-topicness, make a decision from there. If it's receiving votes for the wrong reasons then we want it closed.
FallenAngelEyes continued: If a good deal of the community doesn't understand why it shouldn't be receiving votes, then we likely need to outline and emphasize our policy more clearly.
bwarner answered: I don't think the number of upvotes or answers is relevant. If the question is clearly off-topic, it should be closed. If the community seemed to support changing the rules to allow that type of question, I would start a discussion on meta for people to voice their opinions. If the discussion led to a change in the rules, the question could then be reopened.
Raven Dreamer answered: There has to be something said of examples. Often, when a user gets a question of theirs closed, they will attempt to defend their question with examples of other, similar questions elsewhere on the site. For this reason, even highly voted, mostly off-topic questions need to be closed or migrated, for the simple fact that if they are off-topic, they will probably generate other off-topic questions, especially if they are high-visibility in the first place.
sjohnston answered: If the question obviously belongs on another site, I would migrate. If a reasonable re-wording would make it kosher, I would suggest that in a comment (and edit it myself if the original poster doesn't). If it's somewhat unclear whether it's on-topic or not, I might start a meta discussion to decide if that category of question is appropriate. Once a decision was made on meta, I would follow through, either closing the question or leaving it be. So, TL;DR, it depends on the situation.
Nick T answered: If such a question can be moved to another site (possibly introducing mono-stackexchange users to other sites), do so. If it's borderline but garners significant support, then it's fine by me.