High-net-worth individuals have become more cautious as fears over recession and the impact of high inflation bites.