Class Tags and Diablo 3

Question

In the spirit of things elsewhere on the site, and since the Diablo III promotion is winding down, I thought it might be prudent to reflect on the consensus to allow class-specific tags for a game.

Obviously, I don't expect opinions to have changed much during this time, but the more relevant change is that we now have close to Fifteen-Hundred questions worth of new data to consider.

To reiterate, the consensus was as follows:

These tags add useful information. For example, I am very interested in Diablo 3, but not at all interested in things having to do with the Barbarian. If I mostly play as Witch Doctor and Monk, I can watch those tags specifically and add my expertise to those classes while not being bothered with questions about Wizards, who I don't know about.

And the tags under discussion are these:

• diablo-3-demon-hunter
• diablo-3-barbarian
• diablo-3-monk
• diablo-3-wizard
• diablo-3-witch-doctor

Have we failed? Succeeded? Failceeded?

Answer 1

One thing that I noticed was that there were many class specific questions asked that did not include the class tag. However, of the questions I encountered without a tag, I believe many were asked by relatively new users who may not have known about the tags. (The fact that they show up as suggested tags doesn't really excuse this behaviour, though.) In those cases, where I knew which class they were referring to, I simply added the tag and carried on.

From those questions that did have the tag included, it made the front page/main question page much easier to parse for questions that I may know the answer to. Seeing a skill I didn't recognize in the title was not a problem, as the tag quickly narrowed the scope when looking for additional information from other sources.

For a game that relies so heavily on class specific skills, class tags are a no-brainer. It's obvious to me that they add incredible utility to Diablo 3 questions. I'm still unsure if class tags are a good fit for all class based RPGs (I mentioned the Mass Effect series previously as games I thought wouldn't make for a good fit), but for games with a highly structured and visible set of class specific skills, these tags make sense.

Answer 2

A few days ago I combed through the most recent 270 questions tagged diablo-iii, and identified all questions which had been tagged with such class tags, or had not been tagged with such tags, though they probably should have.

A google doc of all my data collection is here, though the data is already quite old, and Oak has already retagged some of them.

Overview

First, let's get some stastistics from the sample:

% of sample questions with at least one class-specific-tag = ~13%

% of questions within the overall tag with at least one class-specific tag = ~14%

So it looks like we got a pretty reasonably representative sample.

Before we go further, I want to talk about the individual tags themselves. When I first started collecting data, only diablo-3-monk had a tag wiki (though diablo-3-witch-doctor has since gotten one as well).

This is a bad thing.

Without tag wikis, it is less obvious to new users what they should be tagging questions with, or how the tags are supposed to be used (more on this later). Regardless of your views on whether they're working or not, these tags need wikis, a fact made all the more embarrassing since they've already got more questions than other, less useful tags that do.

Sample Analysis

During the approximate 2-day time period in which the 270 sample questions were asked, I found a total of 33 class questions and 9 questions that should have had the tag. My criteria for determining the latter category was simple: questions that referenced a class, class ability, or class achievement by name in the title but did not have the related tag.

This means that about ~21% of all questions that should be tagged are being tagged inappropriately, at least when first asked. (There were also a few questions which did not seem to warrant a tag ((but had it anyway)), but I did not include these in the figure)

And it's not just new users who are not tagging appropriately, as even 10k+ users have left it out of their initial question.

As an aside, I'd be curious to know (and unfortunately, do not know an easy way to compile this info) how many of the class-specific questions have been retagged to add them after the fact. Those ~21% were only the questions that other users hadn't edited already; how much larger would that number be without the diligence of folks like Oak?

Generic Questions and New Users

The saying goes that when you have a hammer, every problem starts to look like a nail. The same goes for the class tags. If you are playing a barbarian, you're more likely to tag your question with diablo-3-barbarian even if class doesn't figure into your question at all, or someone hasn't asked a dual-wield vs. 2-hander question about your class.

Finally, some new users do not understand that simply using the class-specific tag isn't appropriate and we get questions like this that need to be re-edited by users.

Conclusions

I am not really sure how to accurately judge the effectiveness of a tag, but I thought I'd get this data out to more people, and encourage their own reflections.

Answer 3

Perhaps we should go by heuristics for this. The more questions are posted for a game the more tags are acceptable.

I would argue that the race tags for Starcraft are ok, because we have about 1k questions and there are only 3 additional tags.

For Diablo 3 we have about 1.5 k questions and 5 additional tags.

But the whole Street Fighter Series has less than 100 questions, adding about 40 characters would be stupid.

Perhaps we should allow 1 additional game specific tag per 100 questions, as long as it achieves the 13-14% quota Raven Dreamer mentioned. This way we have about 13 questions in this tag. Many real games have less than this.

As a game gets more attention it also gets more difficult to browse through all questions and answers. Sometimes I like to go back and look at old questions and answers for my favorite games, just to check if they are still accurate. Perhaps I can improve them. For Starcraft 2 I would need to look at 34 pages with 30 questions each.