Mr Brown thought the rules would primarily hurt renters.
He said: "If landlords have smaller margins, more cost is going to be put on the renter, whether that be fees or an increase in rent.
"I do think rent in general will increase as there will be less supply."
He went on to say that although the move could help first-time buyers who are currently able to buy, as there would be more houses on the market and less competition from potential landlords, it could hurt future first-time buyers (who are currently renters) as the added costs will hurt their saving ability.
Sam Curtis, regional director at Loan.co.uk, agreed that tax changes had hurt the market and said the difference meant some landlords would actually be making a net loss on property.
He added: "I imagine we will see a lot more landlords unload properties in the next two years.
"A lot of the ones I’ve spoken to have buried their heads in the sand but when it actually hits their pockets, many will probably offload a lot more."
Landlords are also being persuaded to sell now to avoid potentially higher capital gains tax if the country was to have a change of government, according to Kay Ingram, director of public policy at advice firm LEBC.
She said: "A number of landlords have decided to sell ahead of the loss of mortgage tax relief and are keen to realise these within current capital gains tax rules amid concerns that a change of government could lead to higher taxes than the current 28 per cent rate."
But Rachel Lummis, adviser at XpressMortgages, said she had yet to see any evidence of a sell off of buy-to-let stock from their landlords.
She said: "While some have said they are considering it, we have yet to see this being actioned.
"We are seeing more landlords purchase via a limited company now rather than in their personal names which is resulting in landlords with portfolios with a mix of ownership in their private name and ltd company."
Ms Lummis added that the type of property landlords liked — typically a two bed flat — had shifted to more high yielding properties such as HMOs and student accommodation, while many were also looking further afield, out of London and Surrey.
What do you think about the issues raised by this story? Email us on firstname.lastname@example.org to let us know.