Clients complete questionnaires, each answer within the questionnaire is scored and those scores are aggregated, resulting in a final score from one to 10, for example (other systems might use a 1-5 scale or any variant), where one represents the lowest level for risk tolerance and 10 the highest level.
In using these questionnaires, it is easy to think it is simply about the end product: the measurement of someone’s personality or attitude. But the reason most of these questionnaires are psychometric is also about the process of creating a tool to measure someone in this way.
How does the tool interpret and achieve that final measure? Psychometrics is therefore about understanding the tool itself and testing if it is valid and reliable.
Attitude to risk is often stated as a psychological trait, but I would argue it is more complex than that. If the former were true, an individual’s attitude to risk would be identical in different scenarios – for example, when contemplating a bungee jump – and you would be able to infer risk tolerance from other aspects of a client’s lifestyle.
But in that example, I know my attitude to risk when throwing myself off a bridge is very different compared to my attitude to risk concerning my finances.
Therefore, it is not that simple and as a result we need to ensure that we create a tool for measuring exactly what we want it to measure: specifically, someone’s attitude to risk concerning their pensions and investments, and their psychological preference for taking financial risk.
Designing psychometric questionnaires
There are a number of potential pitfalls to using a psychometric questionnaire that designers need to address:
- A questionnaire might not have been built with psychometric testing theory;
- The terminology in questions might be very complex, or assume prior investment or mathematical knowledge (for example, understanding and calculating percentages);
- Questions might ask clients to consider and compare themselves to their ‘peers’, which can be difficult to define;
- Questions might overlap and repeat themselves in some way, which can be frustrating for the respondent and invite problems, because the client naturally answers differently, assuming each question aims to elicit a fresh response;
- Finally, a question set might not capture and measure multiple dimensions of risk and, in particular, ignore the importance of emotions. How does someone feel about risk?
Poorly designed tests with ambiguous wording can lead to a range of measurement errors. If the questions used to make up an assessment are badly worded, inconsistent or confusing, the resulting test will be less reliable. Designers need to think about the questions themselves and how they are constructed: are they open or closed questions, for example?
