How much diversification is worth having?

‘Don’t put all your eggs in one basket’ is one of the more straightforward proverbs to apply to investors. When investing, this suggests one should spread one’s savings into assets which don’t all go up and down at the same time.

But there should surely be a good investment case for each of these investments on a standalone basis, and how much spreading of investments is really worthwhile?