Thurstonian Models - an Answer to Gridgeman's Paradox?
By David Lundahl 1/11/97
One hot topic in the sensory evaluation field is to use Thurstonian models as an alternative to "Fisherian models" such as ANOVA. These generalized linear models take count data from paired comparisons and develop interval scores as an estimate of the "psychological response distance" separating the respective samples. The strength (and weakness) of these models are the models itself and the assumptions associated with these models. Current discussion and use of these models is hottest in the applications to sensory difference tests. Here the duo-trio, triangle, paired difference (e.g. A Not A) and other more elaborate tests are applied to determine if two products are different or the same. Standard Fisherian statistics use the t-test or exact binomial test , where the proportion of correct responses is compared statistically to the proportion of correct responses by chance alone when there is no sensory detectable difference between the samples. As an alternative, the Thurstonian model assumes the underlying responses to each sensory stimuli are each normally distributed, independent and with constant variance. Then it assumes that panelists use specific rules to come up with a response to the sensory difference test (e.g. which sample is most different?"). For some tests like the duo-trio test, the possible decision rules used by a panelist are fairly limited. Other tests like the triangle, have many possible decision rules. A given Thurstonian model must assume that a given panelist uses one decision rule, but which one? There are other problems to contend with, like psychological sources of error resulting from, for example, the order of presented samples in the difference test. These are typically not included inThurstonian models although work as been published in the literature to address this issue.
There is another area of interest pertaining to Thurstonian models. In the literature, you often see citations where Thurstonian models claim to "prove" Gridgeman's Paradox. What is this and what paradox does Thurstonian models explain that other models do not? Well, this addresses some research results where the number of correct responses is observed to increase when a single sensory quality is first identified prior to the evaluation of a sensory difference test (e.g. triangle or duo-trio test). When only one sensory attribute has been identified, the panelist is allowed to focus on that one attribute. It is no paradox that sensory performance increases as the panelist can correctly search for the sensory quality most likely causing the sensory difference. A special class of Thurstonian models has been developed to predict the increase in sensory performance when a panelist is given the correct sensory stimuli to focus on that will result in maximum performance in a difference test. These models define the sensory responses to each sample in a difference test as a multivariate experience. That is, when a panelist is focused on only one attribute the standard Thurstonian models apply since they model sample responses in a univariate dimension. When a panelists has not a clue as to what is the sensory quality or qualities describing the difference, then models have been developed and applied to sensory difference tests that predict sensory response as multivariate normally distributed (e.g. multiple sensory attributes evaluated and remembered in a psychological sense). As you assume more and more dimension, the Thurstonian model predicts the sensory task is more difficult and the expected performance of a panelist decreases. Therefore, if you can assume a panelist cannot remember more than, say, ten dimensions during a sensory experience, a model can be applied that predicts the performance decrease by a panelist.
Now the clincher and point of this article. Just because a model predicts a decrease in performance as you assume more sensory dimensions are used to conduct a sensory task, can you use this as proof that it explains or predicts Gridgemen's Paradox. I think not. For example, consider your own experience and technique in conducting a difference test. Is performance more tied to the dimensionality of the sensory experience or to the ability to correctly identify a sensory difference resulting from some physical or chemical difference in the samples? I surmise the latter is more the case for the following reasons. A more experienced panelist will use various sensory cues and their past experience to develop a strategy that best results in the identification of a key sensory difference. The dimensionality of the sensory experience does dot change. Being told what to look for simply increases the chance that the correct stimulus is used to perform the sensory difference test.
Models are simply our best guess in describing and predicting reality. It is folly to claim a model explains Gridgeman's Paradox simply because it has been observed to predict well in a few carefully controlled sensory experiments. Or put another way. Is Gridgeman's Paradox really a paradox at all? Thurstonian models are useful when the assumptions underlying their makeup are better than other models. Fisherian models (e.g. ANOVA, t-tests) use powerful statistical theory based on assumptions that have been proven to be very robust, e.g. can withstand a diverse array of situations. It is good to strive to develop new models which are more robust or more powerful in certain situation when a different assumption can or cannot be assumed. However, I believe it too early to run out and begin applying Thurstonian models to analyze sensory data.
At InfoSense we are evaluating Thurstonian and other models to identify situations where their assumptions hold up. Jim Kolsky (our Senior Statistician) did his Ph.D. dissertation on Thurstonian models and is an expert in the application of generalized linear models to data. We are open to working with you to help extend your data analysis capabilities and advance this important area of research.