Regulation could be to blame for the current mortgage price war that is pushing down rates and has already caused smaller lenders to leave the market.
Advisers suggested challenger banks found it difficult to maintain a level of competitiveness or make inroads into the market as mortgage rates sit at historic lows and margins continue to be squeezed.
And some said ring-fencing regulations were behind the trend, which saw global banks, such as Santander, HSBC and Barclays, required to ring-fence their retail arm from their investment arm from January 1 of this year.
The BoE itself stated in its Global Developments and Domestic Financial Conditions report, published last month (May 2019), that the "competition in the mortgage market" could have been "amplified" by the ring-fencing of major UK banks.
Last month Tesco Bank ceased mortgage lending and its CEO put the exit down to "challenging market conditions" and "limited profitable growth opportunities".
Nationwide Building Society also saw a narrowed net interest margin which the lender stated reflected "conscious pricing decisions and competition for lending".
Andrew Montlake, director of the broker firm Coreco, said: "With transaction levels painfully low, the mortgage market has never been as competitive as it is right now.
"Margins are being squeezed as rates race to the bottom in a bid for market share and Tesco appear to be saying that the numbers no longer stack up for them."
The Bank of England rules created a firewall between the banks' investment banking operations and their lending arms to ensure consumer banks are still able to lend and consumer money is safe even if a shock hits the banking sector.
With the ring-fence in place, the retail side of the bank is sheltered from the full force of the shock and consumer access to savings, loans, and the ability to pay with debit and credit cards is protected.
But this means that funds which formerly could have been used to back riskier investments are now trapped in the retail environment and one way of banks putting this money to use is to lend — hence, increased offering in the mortgage market.
As the big banks find themselves with piles of cash ready to be lent they "may be incentivised to increase mortgage lending", the BoE has previously stated.
Peter Izard, business development manager at Investec Private Bank, said this had created the "perfect storm" as the "big retail banks are awash with liquidity and there’s not the demand to meet the supply".
He added: "This has led to such lenders increasing their criteria and reducing rates, which is why banks like Tesco can’t compete.
"It’s the perfect storm. Brexit has reduced demand, interest rates are low, and now this massive liquidity for the retail banks has come in at a time when competition is so great."
Mr Izard went on to say that high street banks had the upper hand when it came to economies of scale, which had further added to the drive down of rates seen in the current market.