Just one third of people aged 55 who have accessed their pensions are currently using a financial adviser, Canada Life has found.
Almost 45 per cent of those that didn’t seek advice said they didn’t do so because they had confidence in knowing what they were doing.
One in four said they didn’t believe their pension pot was large enough to warrant financial advice, while 22 per cent were put off by the perceived cost.
According to Canada Life’s research, trust was also a barrier, with 20 per cent of those surveyed not seeking advice due to trust issues.
The insurer had analysed research from Opinium carried out in March among 505 over 55s who have used the pension freedoms.
Andrew Tully, technical director at Canada Life, said: "A significant majority of people exercising their pension freedoms have turned their back on financial advice.
"We’ve exposed some pretty entrenched views on the value of financial advice, but many people are seemingly making complex decisions and potentially irreversible financial decisions unaided.
"I’d like to turn the debate on its head. Rather than talk about the cost of advice, we should instead promote the idea that people can’t afford not to get advice. It can often be just as cost effective, or even better value paying for advice than not."
The research also found those who used a financial adviser were twice as likely to withdraw a cash lump sum and buy an annuity with the balance of their pension than those who didn’t get advice – 37 per cent compared with 18 per cent.
The regulator has in the past voiced concern that people were accessing their pensions and defaulting into low yielding cash accounts, potentially missing out and forfeiting later life security.
Patrick Connolly, chartered financial planner at Chase de Vere, said: "It is worrying that two-thirds of people are accessing their pensions without taking financial advice.
"The flexibility offered by pension freedoms can provide excellent financial planning opportunities.
"However, the decisions that are made when accessing pension benefits are too important to get wrong and if people make the wrong choices this could impact negatively on their future standard of living."
Scott Gallacher, chartered financial planner at Rowley Turton, added: "I’m not surprised by the findings of the research.
"Many people underestimate the value of advice whilst over estimating their own ability. It’s very rare that advisers can’t add value to people’s retirement planning, although for smaller pots that value can sometimes be less than the cost of advice."
He said he had come across a client who was initially reluctant to commit to the cost of his advice. But in the end he was able to show him that he could match his retirement income (annuity) and increase his tax free cash lump sum by more than £20,000.
