Continued consolidation in the adviser market could lead to the fund space being reduced to a small number of "mediocre" providers, Hargreaves Lansdown founder Peter Hargreaves has warned.