Measuring Confidence. Risk and Uncertainty

Introduction. The development of modern science is marked by our ability to express certain phenomena in a numerical form - to measure. We will not discuss here the validity of this approach but simply accept the fact that without assigning numerical values, science apparently cannot progress in analysing any given phenomenon beyond the contemplation of its complexity.

Confidence (specifically trust) escapes clear measurement for several reasons, whether it is its unclear definition, its complexity, subjectivity or the fact that it is a mental process, unavailable to instruments. There is a strong argument [Castelfranchi2000] against the use of simplistic metrics to measure trust that validly points to the complexity of the process and the embeddedness of trust in so many instances of our activity that it may be impossible to capture (and measure) trust in a clear form. If we consider also that trust is one of the sub-constructs of every utterance [Habermas1986] (so that we cannot rely on language to be trust-neutral), the task of measuring trust, control or confidence seems daunting indeed.

However, even simplification of a natural phenomenon has a value as a tool that guides our understanding and sheds light on interesting aspects that would not be discussed otherwise. Assuming that confidence can be captured by a single number (or a small tuple of related numbers) is a simplification, but it may turn out to be useful as it opens up opportunities to investigate and model certain aspects of confidence such as cooperation, propagation, etc. We should only understand that what we measure may not be exactly trust, confidence or trustworthiness, but something (hopefully) very close to it.

Beginning with this warning, this chapter explores various ways in which confidence can be captured in its numerical equivalent and processed within the context of the model. We start from a discussion of risk and uncertainty. From there we will investigate some empirical methods that can be used to assess the extent of trust and trustworthiness (that seems to be harder to capture and more interesting than control itself).

This is followed by a discussion about different metrics of confidence, not only from the perspective of their numerical range, but also in relation to their underlying assumptions. Those considerations lead to the discussion of different operations that can be performed on confidence.

Risk and Uncertainty. We can see from the model that the situation related to confidence is characterised by two elements: trust and control. We can also view it as being characterised by two different concepts: uncertainty and risk. There is an observation about the difference between risk and uncertainty ([Ormerod2005], following [Knight1921]) that the situation is characterised by risk if the outcome is not known but the probability function of the outcome is known.

That is, while an individual transaction may end in success or failure, when repeated a sufficient number of times, we can see certain known distribution of outcomes (e.g. 10 per cent of transactions fail). Following this reasoning, the lack of confidence associated with situations that are characterised by risk can be offset by known methods (insurance, securitisation, diversification, etc.).

In contrast, situations characterised by uncertainty are those where the probability function is not known - or is not relevant. Such lack of knowledge can be attributed to several factors such as insufficient number of samples, lack of access to important information, inability to process information, irrelevance of personal experience, etc. Whatever the case, uncertainty cannot be offset by any instrument exactly because it is not known how such an instrument may operate. Ormerod argues that in an increasingly complex world the situation of uncertainty becomes prevalent, specifically at the individual level of experience.

The perception of a duality of risk and uncertainty is further reinforced by the identification of two types of 'uncertainty' (the terminology may be slightly confusing here) [Helton1997] (see also [Dimitrakos2003] for an alternative approach). The first type of uncertainty (aleatory uncertainty) is associated with the potential random behaviour of the system. Such uncertainty may randomly affect observations and can be best handled by probability theory.

In contrast, the epistemic uncertainty (the second type of uncertainty) is associated with ignorance, bounded rationality and subjective inability to reason. Epistemic uncertainty cannot be reduced to probability (it is unlikely that it can be reduced at all) but may significantly influence the reasoning. For clarification, what we call here 'uncertainty' is the uncertainty of the latter kind, the epistemic kind while the concept of risk is closer to the former.

Relationship to the Model. Drawing the parallel between this reasoning (of risk and uncertainty) and the model of confidence, we can say (simplifying) that the control path is helpful in dealing with situations that can be characterised by risk while the trust path addresses situations that are related to uncertainty. It does not imply that we rigidly equate risk with the control path and certainty with the trust path.

Situations seem to usually have elements of both, with uncertainty being more pronounced where there is little structure, experience or knowledge and risk is visible in structured, routine activities. However, we may consider risk a good method to express control (with consideration to the note below) and certainty a good method to express trust.

The control path is actually more complex: it is built on risk, on the assumption that the probability (or the probability distribution) is known. However, the fact that we use control instruments means that there should also be the element of confidence (ultimately trust) in such instruments. Therefore, the control path works on the basis of uncertain probability, i.e. on the construct that allows expression of probability of the assumption that the certainty has been achieved about the instrument.

Example. Let us explore a simple example. Let's assume that Alice would like to invest a certain amount of money in the fund that is managed by Bob. She has done her research on funds in general and Bob's fund in particular and she believes that she has a good understanding of what kind of performance she can expect - this is her risk-based probability distribution.

However, she also faces uncertainty associated with her research - whether she has used reputable sources, whether Bob has cooked his books, or not, etc. Considering the control path (and heavily simplifying the story), Alice is uncertain about her risk distribution. She believes that she can safely predict Bob's performance only if her sources are reputable.

She also faces another uncertainty, this time one not associated with Bob's performance. At any moment Bob may execute an exit strategy, close his fund and disappear with all her money. This uncertainty has nothing to do with his past performance and cannot be estimated by probability distribution. It is only Bob who can decide whether he is going to default on his customers or not.

Even knowing the probability distribution of the behaviour of the population of fund managers will not help Alice - Bob is the critical element (her single point of failure) and Alice cannot protect herself against his default. The situation will be different if Alice diversifies her investment into several funds, or takes out an insurance, moving also this part of her decision into the control path, but in that case she will probably be uncertain about someone or something else.