Empirical Metrics. Surveys. Games of trust

Empirical methods try to capture the value of confidence by exploring the behaviour or introspection of people, to determine the perceived or expressed level of confidence. Those methods combine theoretical background (that determines what it is that they measure) with a defined set of questions and statistical processing of results. Alternatively [Elofson2001], intelligent agents can be used to elicit and then replicate human decision-making process

The outcome of empirical measurements is essential to verify a hypothesis, and it serves as the ultimate reference point in simulating human confidence in artificial environments. At the same time, they suffer from common problems such as the relevance to the underlying hypothesis, cultural dependence, language embeddedness, representative selection of participants, etc.

The methodology of empirical measurements has been developed to counteract and minimise those negative factors. Such a methodology usually concentrates on those elements of trustworthy behaviour that are (according to underlying theories) clear signs of trust. The willingness to cooperate and actual cooperation are commonly used as an evidence of and a measure of trust. The actual value (level of trust, level of trustworthiness) is assessed from the difference between the observed behaviour and the theoretical behaviour that should have been observed in the absence of any cooperation.

Surveys. Surveys attempt to capture the level of confidence (and are mostly concerned with trust) by means of both observations or introspection, i.e. without engaging in any experiments. Respondents usually provide answers to a set of questions or statements and responses are e.g. structured according to a Likert scale. Differentiating factors are the underlying theoretical background (what they measure) and contextual relevance (in what situations they are applicable).

There are several trust surveys with their associated scales, designed for different purposes. One of the earliest ones (yet still in use) is McCroskey's scales [McCroskey1966] that have been used to determine authoritativeness (competence) and character (trustworthiness) of speakers. The authoritativeness scale consists of 22 questions with responses selected from a Likert scale and contains statements designed to elicit the respondent's opinions and beliefs regarding different aspects of the speaker's competence, along six main semantic differential scales such as 'informed-uninformed'.

The character scale consists of 20 statements, again used to gather opinions and beliefs, this time about the trustworthiness of the speaker, along another six semantic differential axes such as 'unselfish-selfish'. Original scales have been thoroughly verified against the set of profiles using a large group of respondents.

Of others, Rempel's trust scale [Rempel1985] and Rotter's scale [Rotter1971] are quite popular in determining the level of interpersonal trust in different settings. For a particular research area a more specific survey can be developed. For example, the interdisciplinary model of trust [McKnight2001], [McKnight2003] has been verified using a survey while [Corritore2005] uses a survey to establish the relationship between design elements of the website and perceived trustworthiness of it.

The Organizational Trust Inventory (OTI) [Cummings1996] is an example of an exhaustive, theory-driven survey that can be used to determine the level of trust within the organisation. The underlying theory defines trust as a collection of three dimensions of beliefs, each dimension assessed on three components: affective state, cognition and intended behaviour.

The level of trust can be estimated as the difference between the actual organisational behaviour and a theoretical behaviour that is characterised by an absence of trust. The survey contains 81 statements and responses can be related back to original nine factors. There is also a shorter version of the survey that contains only 12 items that is similarly reliable but can be administered and decoded in much shorter time.

An alternative to a survey is a one-side bet that allows for the quick estimation of a person's subjective probability of a transaction. Assuming that the potential gain is fixed, the amount that a person bets can be used to estimate his confidence. If a game is open-ended but a person has limited resources, the amount that is being bet can be used to indicate the level of confidence in the situation.

Games of trust. Another method to empirically measure confidence (again, we are here mostly concerned with trust) is to engage participants in experiments and to treat the outcome of such experiments as an estimate of confidence. Several economic games and game-like scenarios have been tried, with certain preferences for those that allow the estimation of confidence in monetary terms (see [Keser2003] for an interesting overview).

Games of trust are designed in such a way that their Nash equilibrium differs from the Pareto optimum so that no player alone can maximise his own utility by altering his selfish strategy without cooperation, while cooperating partners can benefit. However, these games are not strictly cooperative, as the communication between players is restricted.

Therefore any appearance of cooperation can usually be attributed to trust (in some games there is an element of control involved). Trust can therefore be estimated on the basis of monetary gain on cooperation and then expressed e.g. as a fraction of the difference between the gain from the behaviour driven by maximisation of individual's utility and behaviour where the cooperation maximises the common utility.

There are several advantages of using games to estimate the level of trust or confidence. The familiar yet fictional environment makes participants engaged but less concerned about their real-life status. Controlled conditions of the experiment allow several variations (anonymity, iterations, etc.) while the controlled pace of the game may inhibit overly deliberate responses for the benefit of more intuitive-based ones. Finally, money acts both as a familiar but emotional stimulus and a simple numeric estimate of the outcome.