I'd say, like most things, it depends.
If you searched the site and/or google and didn't find it, and the words you used are likely to be used by others who are searching for that same topic, then you're adding value, and it may as well stay around.
If, after looking at the duplicate you see some or all of the following:
- You've asked almost exactly the same thing, and just didn't think to search (or made a mistake searching, etc)
- There are other duplicates already (ie, it's a popular topic that many people are asking about and creating duplicates for)
- There's nothing "special" that makes your question interesting or new (ie, screenshots or specific details, different language, etc)
... then there's little value in keeping the duplicate around, and you may as well delete it.
I tend to err on the side of keeping duplicates around, since usually there is little to no "cost" to keeping them, and they can sometimes serve as useful signposts. However, in situations where the new duplicate is a repeat of a common question or particularly low quality, I have been known to remove them.