He also said that with the recent changes to the buy-to-let sector, the market was unlikely to see a "property elite" who owned all the property while the rest of the country rented from them.
He added: "I would have been worried about that four or five years ago, but the slowing down of people entering the buy-to-let sector and the changes in tax and regulation mean this is probably not an issue anymore."
Landlords have been subject to a number of regulatory changes in recent years, with the introduction of an additional 3 per cent stamp duty surcharge on second homes in April 2016, followed by cuts to mortgage interest tax relief.
In order to help the housing market keep moving, Mr Morrey said a tax break for people downsizing and passing on money to their children would help shake up the stagnant position.
He said: "We need some kind of help for second steppers, which is hard to come by, but freeing up more property in the upper end of the market would only be a good thing.
"We need to keep building houses, incentivise downsizing and try and help those second steppers that would shift the market along."
Earlier today, the secretary of state for Housing, Communities and Local Government, James Brokenshire, said first-time buyers should be able to use their pension pots to top up their savings for a house deposit in an effort to help those unable to make the first step onto the housing ladder.
What do you think about the issues raised by this story? Email us on email@example.com to let us know.