# Method of scoring and the law of categorical judgments, Obtaining...

## Getting scores

One of the most well-known and common procedures for indirect scaling is the attribution of stimuli of any order values ​​to the test subjects in accordance with the numerical or verbal categories assigned to the experimenters. This method is known as the scoring method.

It is generally accepted that this method originates in the works of the English naturalist, one of the founders of the psychology of the individual differences of F. Galton. Back in the early 80's. XIX century. he asked his subjects to assess the degree of clarity of their mental images using a point scale. However, the scoring procedures themselves, of course, have an even longer history, independent of the history of psychology and psychometrics.

Apparently, the most well-known scoring scale is the school achievement scale, which uses five categories for assessing a student's achievements. Note that such a scale does not necessarily have to use numeric values. So, instead of numbers in the scoresheet, you can use such characteristics as "excellent", "good", "satisfactory", "unsatisfactory". Each of this verbal category can be associated with a numerical value: 5, 4, 3 and 2. It is this way of denoting the grades of academic achievement in the practice of higher education in our country.

Thus, we see that the scoring method involves the allocation of a set of quantitative categories set by the experimenter, between which order relations are established. These categories can be given by both numbers and verbal names. There is also a graphic version of this method, when the relationships between numerical or verbal categories turn out to be clearly defined in the form of horizontal or vertical segments.

Often, each scale graduation is indicated by the researcher separately. For example, when measuring the sensation of the weight of objects, you can specify the following categories of scores: 1 - "very light", 2 - "light", 3 - more likely easy, "4 -" rather heavy ", 5 - , heavy ", 6 -" very heavy ".

The scores obtained from the test subjects are averaged. If in the experiment the researcher used only verbal designations, they must first be translated into numerical values. The same is true with respect to the use of graphical scales.

As a rule, the processing of the obtained data is reduced to calculating the arithmetic mean for each scoring of the stimulus to all subjects. Quite often, however, instead of the arithmetic mean, the value of the median is used, since the estimates obtained from the test subjects can be characterized by pronounced asymmetry. The use of the median is more justified if one takes into account the fact that the estimates that the researcher receives from the subject are specified only in the ordinal scale, although the measured continuum can also be expressed in a scale of equal intervals.

## The law of categorical judgments

In spite of the fact that directly the method of scoring gives only an ordinal scale, in principle, it is possible to present the data obtained in the application of the procedures of this method on a stronger interval scale. This possibility, in particular, is given in the law of categorical propositions proposed by Thorgerson.

This law as a whole reproduces the logic of the reasoning of the law of comparative judgments of Terston, considered by us in the previous paragraph. It is also assumed that every stimulus causes a stochastic process of discrimination. This process can be described in accordance with the law of normal distribution with parameters unknown in advance to the researcher - mathematical expectation and variance. When evaluating a stimulus, it is compared not with another stimulus, as it does in the method of paired comparisons, but with the boundary of one of the estimated categories. The sense of this boundary is associated with a separate process of discrimination. It can be described in the same way as the process of distinguishing each individual stimulus is described.

In other words, Thorgerson suggested that the psychological continuum is divided into a number of ordered categories, or steps, determined by the experimenter. The boundaries between the categories turn out to be unstable. Their location should be described using the normal distribution function. Moreover, the boundaries between different categories are characterized not only by different values ​​of mathematical expectation, but, possibly, by different variance values. The comparison of this stimulus with the category boundary is carried out in the same way as a comparison of the two stimuli. Consequently, all the reasoning considered by us in the exposition of the law of comparative judgments can be transferred to the situation under consideration now.

Suppose we have some stimulus S. It, as we know, causes a process of discrimination that is characterized by the mathematical expectation m s and the variance σ s . Suppose the experimenter is given only two categories of answer - A and B. Thus, the psychological continuum associated with the process discrimination of incentives will be divided into two parts. However, the boundary between them is not fixed. Non-location can be described by a normal distribution function with parameters AB and σ A sub> to (Figure 8.3). If the process of discriminating the stimulus gives a value of the sensation that is less than the sensation corresponding to the process of distinguishing the boundary between the categories A and B, then the subject chooses the category A. Otherwise, the category B

Obviously, the frequency of choice of each of these categories may indicate how far from the category boundary is the incentive. If we convert these frequencies to the corresponding values ​​ z , just as we did using the law of comparative judgments, we can get the position of the stimulus on the scale, and the scale itself will be a scale equal relations. Having done this procedure for each evaluation category and then averaging the obtained values, we get not an ordinal, but an interval scale.

Fig. 8.3. The distribution of incentive discrimination and the boundaries between the categories A and In when choosing an evaluation category in the scoring method according to the law of categorical judgments of Thorgerson

Let's explain the described idea on a concrete example. Suppose we have estimates of five objects made by 41 subjects on a five-point scale (Table 8.11).

Table 8.11

The frequency of selecting each category of the five-point scale for the five evaluated objects

 Objects Rating Categories (points) 1 2 3 4 5 1 3 7 9 13 9 2 2 3 15 16 5 3 4 6 16 10 5 4 10 7 15 6 3 5 20 9 2 9 1

We already know that the usual way of processing this kind of data is to calculate the average of the arithmetic or median. However, this method gives us only an ordinal scale. To get a stronger, interval scale, we translate the results into probability values ​​by dividing each value in the table cell by the number of subjects, i.e. 41. Then calculate the accumulated probabilities for each category of point scale for each stimulus. The results of these calculations are presented in Table. 8.12.

Table 8.12

The accumulated probabilities of selecting the categories of the five-point scale for the five evaluated objects

 Objects Rating Categories (points) 1 2 3 4 5 1 0.073 0.244 0.463 0.780 1 2 0.049 0.122 0.488 0.887 1 3 .098 0.244 0.634 0.887 1 4 0.244 0.415 0.780 0.927 1 5 0.488 0.707 0.756 0.976 1

Now we need to translate the obtained probability values ​​into z-values. The results of these calculations are shown in Table. 8.13. The values ​​obtained reflect the distances of each stimulus from each category of the five-point scale. It can be seen that in relation to the category of the scale in 1 point, the object closest to number 5 is located, and the farthest from this category is the object under number 2.

Similarly, we can estimate the distance of the boundary of each category with respect to all objects evaluated. To obtain the scale values, it is necessary to subtract from the total average the mean z-value for each stimulus. If the researcher does not want to deal with negative values, you can easily get rid of them by taking a minimum scale value for zero. In our case, object No. 5 has a minimum value of -0.75. Adding to each value on the scale 0.75, we get only positive values.

Table 8.13

Results of z-transformations of accumulated probabilities

 Objects Rating Categories (points) The value on the scale 1 2 3 4 Average 1 -1.45 -0.69 -0.09 0.77 -0.37 0.41 2 -1.66 -1.17 -0.03 1.17 -0.42 0.47 3 -1.30 -0.69 0.34 1.17 -0.12 0.16 4 -0.69 -0.22 0.77 1.45 0.33 -0.29 5 -0.03 0.55 0.69 1.97 0.79 -0.75 Average -1.03 -0.44 0.34 1.31 0.04

Obviously, quite often some categories of scores will be absent in the judgments of a group of subjects, especially in a situation where the number of subjects participating in the study is small. Thus, the probability of selecting these categories will be zero. (As we recall, this results in empty cells in the matrix of z-values.) Therefore, very often the described method of obtaining an interval scale with scores is supplemented with the algorithm of Torgerson described in the previous section, which allows interpolating the missing matrix values. Such calculations are carried out both for the scale categories themselves and for the evaluated objects.

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

[...]

## Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

## How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)