There's a proposed FAQ on Meta Stack Exchange precisely about this topic. @arghtype's answer summarizes the main differences.
when should I consider using each of them?
You should use the 'follow' feature when:
you want to receive inbox notifications
you want to keep the fact that you're following a post private
you want to keep a reference to an answer,...
My fault. Isn't it always? We had a problem with badges failing to award last night, so I added some extra logging to capture more info when it stumbled. In the process I accidentally broke something such that it appeared to work, but really didn't actually do anything. Deploying the fix now. Sorry folks.
See The Follow Questions and Answers feature is now live across the Network .
Followed questions/answers will generate notifications in top bar similar to answers to your questions or upvotes.
Followed questions/answers are private to you.
Bookmarked questions are public and anyone can check which questions you bookmarked on your profile page....