Identification questions are a poor fit for the StackExchange network, and actively play a part in bringing down the overall quality of the site. On Gaming specifically, they tend to take the form of Identify This Game (ITG), where a user will recite some details on a game he or she once played in hopes of coming away with a title, usually to be able to play the game again.
This is a practical problem to have, and one that can be very satisfying to get an answer to. I don't want to come across like I don't want people to get the answers they deserve for their questions, but we have to recognize that we have been given a very specific framework to work with, and that we should play to its strengths while recognizing its weaknesses. Not every site can do everything, and we should strive to make the best site that we can within the framework that we are given.
What is StackExchange, anyway? What is StackExchange supposed to be? Put simply, StackExchange is a place where experts can use their experience and expertise to help people solve their problems. The goal of StackExchange is to make the Internet a better place by providing free, high quality information. In order to do this, we must attract experts to our site, have them respect our site, and have them stick around and continue to contribute to our site. This is much more likely to happen with a higher average question quality. If removing an entire category of question would increase the overall quality of the site, it's only common sense that we should do that.
It is my firm belief that ITG questions fit exactly into that category. The banishment of these questions will do much more good for this site than their continued existence. Many people claim that they provide value for a variety of reasons that I will address shortly, but I have not seen it. They claim that they can be cleaned up and keep the average quality of the site high, but I have not seen it. This essay is about the current state of ITG, not what ITG could be. If you feel strongly that ITG should be allowed to stay, now is your opportunity to prove it. Show how well you can moderate the bad questions and how you can keep the quality of this site high while still keeping them around.
Let's start with an analysis of the StackExchange framework, and how exactly ITG tries to fit into that framework. Votes are perhaps the single most important thing in the framework. Any user with sufficient reputation (and the threshold is very, very low) is able to upvote any content that they find useful, and downvote any content that is not useful. For answers, this naturally creates a form of sorting, sifting the best answers to the top and leaving the lesser ones further down. It was specifically designed this way. It didn't happen by chance; it was a conscious decision. That's how it's supposed to work.
What happens to ITG in this case? You can't vote on the correct answer unless the asker has already deemed it the correct answer. That's a fact, plain and simple. You can't possibly know whether an answer is correct until the asker has marked it as such. You aren't in the asker's mind, and unless you purport to have a catalogue of every single game ever, with every single quality that all those games have ever had, you cannot give a definitive correct answer without the asker's approval. So upvotes can't be used for correct answers.
What about if upvotes are used on answers that fit the criteria of the question? They answer the question as asked, so that should work fine and dandy, right? Except now we're upvoting answers that might be wrong. That doesn't play into the framework we have very nicely at all. What if the asker made a mistake, and included a false detail? If he doesn't remember the name, chances are he made some other kind of mistake too. Now the correct answer might not fit the criteria at all. Downvoting the correct answer is now the proper course of action if we use this scheme. That doesn't make any sense at all. As of yet, I haven't seen any other propositions on how voting should work for ITG questions. There doesn't seem to be a good conclusion to this problem; it is left open still.
I have seen arguments that we don't need to have conventions for voting. Votes are to be spent as the user sees fit, and that's that. While it is true that we cannot control how one spends his votes, it is not true that we should just fling them about haphazardly and protect that right come hell or high water. We should educate people on the proper usage and importance of votes, so that they can wield them like a scalpel, not a sledge. There is no finesse or order to voting on ITG questions, and an argument claiming that there's no order to voting in general is simply incorrect. Sure, there are outliers that get a ridiculous number of votes because of a meme, but for the most part, votes are how we tell quality from rubbish. On ITG questions, votes are meaningless, and do not help us draw this distinction.
Let's go in another direction. Some people claim that ITG questions indeed can help attract people to the site and keep them here, just as I said was the main purpose of the StackExchange network. If there's a chance that someone can see and participate in an ITG question to start off on the site, and then they can stick around to contribute to the other, more meaningful areas, then that question should be allowed for that purpose. We've had ITG questions since our inception, and I can't think of a single case where this has happened. Yes, the users might stick around a bit and contribute another question or answer, but which of our expert users (remember, the main goal is to attract experts) started on the site as a result of ITGs? According to research done by agent86, more than half of the total ITG questions on the site were asked by people with 150 or less reputation. That amount of reputation can be gained in an afternoon with barely any commitment. The number of quality users that ITG attract is low. While this doesn't disprove the usefulness of maybe, maybe attracting a good user once in 1000 questions, it certainly doesn't give me any cause to hesitate in banning ITG.
Another angle proponents take is that ITG questions help the Internet as a whole, part of the StackExchange network's mantra. People can find their games more easily because we can serve as a catalogue of these kinds of questions on search engines. But the data says otherwise. Again, agent86 ran the numbers, and existing ITG questions have an average of 264 views each. Even Nethack, a niche game that can only be enjoyed by a very select subset of gamers, has almost 50% more views per question at 392. For ITG being our fifth most popular tag on the site, it doesn't even come close to breaking the 1000+ views per questions on the top four. There are many, many ITG questions, but not a lot of people who are interested in them. So they don't attract very many new users, and the users they do attract generally don't contribute much. These are not reasons to remove ITG on their own. But these metrics show that these also cannot be used as legitimate reasons to keep ITG around. The math just doesn't add up.
In the history of Gaming, we have had almost exactly this same argument before. Game recommendation questions had a lot of the same problems as ITG questions. Voting on those kinds of questions was meaningless and random. Game recommendations, however, were also subjective, and didn't meet the criteria for being good subjective questions. In theory, ITG questions have a single, concrete answer, but in practice they play out much like game recommendations. There is speculation abound, and it has even tricked some people into thinking that ITG questions themselves are subjective. They are not. People have defended ITG from the six points that make a good subjective question, but that's hardly relevant. ITG questions are objective. Remember, being objective doesn't automatically make a question on topic. There is such a thing as bad objective, just like there is bad subjective. "How many fingers am I holding up?" is an objective question, too. That's clearly not a good fit for any kind of expert site.
Let us look at these arguments for how ITG questions are good subjective, however. Even though they do not apply, the pro-ITG crowd has spent time explaining how they do. So let's go through and address some of these. We can address the first two points together, because the rebuttal to those is the same.
Great subjective questions inspire answers that explain "why" and "how". Great subjective questions tend to have long, not short, answers. As is readily apparent, ITG immediately fails both of these requirements. Every single ITG answer can be written in 4 words: "Maybe it is [game]." Any extra information provided in the answer is fluff, and not part of the core answer. There is nothing to explain in an ITG answer, either, so the first point is also failed. So what's the rebuttal? "In some cases, this criterion does not apply." What? Why not? You can't just ignore criteria. The very fact that you have stooped to saying that the criteria shouldn't apply is itself an admission that the question fails to qualify for what you're trying to prove. Just because you say the criteria aren't important doesn't make it so.
Great subjective questions are more than just mindless social fun. ITG questions are the epitome of "Getting To Know You" (GTKY) questions, which have been rather scorned across the StackExchange network for their failure to work well within the system. We are not a discussion forum. We have a community, yes, but the primary function of the main site is to provide high quality questions and answers. If you want to get to know each other, use the chat. To quote Matthew Read in response to this point: "I don't think indirectly finding out what game someone used to play is any different than finding out what game they currently play when they post a strategy question or a how-do-I-beat-this question." But these are completely different. Indirectly finding out information is just a part of the network. On StackOverflow, merely asking a question about a language indicates that you're using that language. It doesn't make it a GTKY question any more than asking about StarCraft 2 strategy. But ITG questions take it a step further. The question itself becomes a kind of social forum, where people put forth their best guess and discuss the merits of each answer. It is not like a strategy question, and cannot be compared to a strategy question.
The other three points hold up, but that's not enough. A good subjective question must meet a majority of the criteria. Most ITGs meet three of those, which is a mere 50%, and not enough to really warrant keeping. Even if they aren't treated like GTKY questions, that's still only two thirds, and a weak case at best for an argument that doesn't even make sense to begin with.
So why not just keep the "good" ITG questions? Let's define what makes up a good ITG question. We currently have a status quo where ITG questions are only allowable if they are sufficiently detailed. The thought is that the more detailed the question is, the more likely we'll be able to get them a correct answer in a reasonable amount of time. That's not necessarily true, though. More detail might be meaningless if you spent all of your time playing part of the game that no one else was interested in. I can't do this point as much justice as agent86 did, so check out his answer. The main takeaway is that there is no way to tell helpful and unhelpful detail apart. So far, we haven't come up with another way to tell "good" ITG questions from bad ones, but I challenge any of you to come up with a scheme.
Another thing to take into consideration is that we have allowed ITG since the site's inception, and it doesn't really work. ITG questions very easily turn into broken windows. Broken windows are very low quality content that will be perceived as trashy and turn new users away if they encounter them first. Compare to driving through a neighborhood and noticing broken windows on some of the houses. You're going to get out of there right quick, because you don't know what goes on behind those broken windows or what broke them, but you don't want to stay. The same is true of websites that have egregiously low quality content, like ITG is prone to do when left unattended.
So far, no one has stepped up and played an active role in helping to rein in these broken windows until LessPop_MoreFizz catalogued the absolute worst ITG questions, challenging a moderator to do something about it. There are still questions left that will act as broken windows, though, and more are getting added daily. This takes away from what was identified as the main goal of the StackExchange network early on in this essay: to attract users. Without proper moderation, the vast majority of ITG do exactly the opposite. No one stepped up to do this task until specifically called out. No one has volunteered. Even if someone did, I expect it would be met with grave difficulty, as the line between what makes a good question and a bad question is blurry at best.
As yet in this essay, I haven't said much about the quality of answers. Even with all the evidence presented above, the most damning thing about ITG is that it's all speculation. We are a community of experts, who use our expertise to solve gaming problems. When speculation is involved, the expertise goes out the window. Average Joe's answer is as good as anyone's. That's not to say that Average Joe can't participate in the site, but that Average Joe should at least have a modicum of experience in what he's talking about to contribute. Answers to ITG questions are guesses. There are no two ways about this statement.
When answering an ITG question, the best you can be sure of is that your game meets the criteria set forth in the question. You can be 100% confident in that, and you can be correct in that. But that 100% certainty that your answer meets the criteria set forth in the question does not correlate to a 100% chance that your answer is correct. It is impossible to answer a question truly knowing that you are correct. You are putting a guess out there, hoping that it is what the asker was talking about. For every accepted answer someone points out that was the "obvious" choice, I can point to five answers that were wrong.
What in the world are you doing on a site supposedly full of experts flooding questions with wrong answers? How can you possibly answer a question knowing full well that your answer might be wrong? That goes against the entire spirit of the site, the entire network, and the Internet as a whole. We strive to make the Internet a better place, to produce high quality content and answers, and speculation of the sort in ITG just spits in the face of all of that.
ITG is not inherently evil. There are good ITG questions out there that deserve answers. But that does not mean that our site is the place to go for those answers. We have plenty of knowledgeable people in chat who are willing to help you find that game you're looking for. And if that doesn't work, then maybe this site isn't for you. Maybe there's another site out there that will help you with all your recommendation and ITG needs, but it's not us, and it's not our job to find it for you. We're just experts, providing expert information and bettering the Internet. And we'll do that a whole hell of a lot better if we can stop arguing about ITG and move on to more important things.
The number of users who had one of their first two posts (Answer or Question) in ITG and have not gone on to make more than 1 UpVote: 185 (54%). You can see my research for comparison to other questions