Some victims of pension scams have lost more than £1m in savings to fraudsters, figures from government agencies have shown.