The panel, which is an independent consumer body to advise the Financial Conduct Authority, carried out research which found most people do not read terms and conditions in any detail - at best skimming them - and instead rely on the "wisdom of the crowd".
Even when they did read the terms and conditions on data, only a small number of those surveyed were able to answer questions on them correctly, with these results barely improving when they were given a second opportunity to read them.
Moreover the research found consumers do not understand the value of the data they are allowing companies to use, and in general considered the delivery of more targeted advertising to be a fair use of data.
The findings are timely because the General Data Protection Regulation (GDPR) which comes into effect on 25 May, requires consent for a company's use of data to be "freely given, unambiguous and informed" but the panel said its evidence suggests this is not often the case.
In addition, the roll-out of Open Banking and the Second Payment Services Directive mean consumers can share access to their bank accounts with third parties to open up the market to innovation and competition, which paves the way for a host of new services based on consumers' financial data.
Sue Lewis, chairman of the panel said: "Our research underlines the need for genuinely informed consent. Ensuring people understand what they are signing up to is vital.
"For too long, terms and conditions have been written to protect firms and not to enlighten consumers. That has to change. People also need to know which providers they can trust, and what happens to their data.
"There is an opportunity for innovators to provide better services for consumers, and to spur change in the wider industry. But the government and the FCA need to get consumer protection right now, not wait until things have gone badly wrong and trust in the new world has been undermined."
The panel made a number of recommendations for the industry, as well as the FCA and the government.
It called on all stakeholders to find an alternative to lengthy and complex terms and conditions, taking account of behavioural biases. It also called on the FCA to introduce a requirement on all firms using consumers’ payments data to have ethics committees to oversee the governance, use and sharing of that data.