Reliability assessment procedure, Procedure "same-different"...

Confidence Assessment Procedure

This method is economical and can be used in situations where the researcher is required to obtain several points of the RCP curve in one experiment that preserves stable conditions for producing empty and meaningful samples.

The confidence assessment procedure is similar to the "yes/no" procedure. The subject is also given a large number of empty and meaningful samples. The difference is that the subject must evaluate how confident he is in his answer. It is assumed that n gradations of the confidence scale give n - 1 positions of the decision criterion: from an extremely liberal criterion that divides the answers into one point, on the one hand, and two points or higher, on the other, to the extremely conservative, dividing the answers into n-1 points and lower, on the one hand, and n points on the other.

For example, you can ask the subject to use a five-point scale:

1 - definitely was not;

2 - probably was not;

3 - I'm not sure if it was or was not;

4 - rather it was;

5 - it was quite accurate.

Thus, four decision criteria will be given: C {, C 2 , C 3 and With 4 . If the intensity of the sensory impression exceeds the criterion value C 4 , , the subject will select the answer "5". If the sensory impression is higher than C 3 , the answer "4" is selected below C 4 , . In the case where the intensity of the sensory activity is higher than C 2 , but below C 3 , the answer is "3" . If the intensity of the sensory activity is higher than Cj, but below With 2 , the subject will select the answer "2". Otherwise, if the intensity of the sensory experience does not reach the value C j, the answer is selected 1 (Figure 7.7).

Fig. 7.7. Selecting the answer depending on the position of the criterion in the confidence assessment procedure

The processing of the results begins with an estimate of the number of coincidences of each category of the subject's response with the presentation of meaningful and empty stimuli. These estimates are considered as hits and false alarms for each answer. With this approach, it is obvious that all the answers of the subject are examined by analogy with the answers "yes" in the yes/no method.

Then, based on this data, the probabilities of hits and false alarms in the subject's responses for each position of the decision criterion are evaluated. For example, the probability that a subject will choose a response from one to five points is obviously 100% for both hits and false alarms. For the next value of the decision criterion, the likelihood of hitting and false alarms can be determined by summing the probabilities of responses corresponding to estimates of two or more points. Similarly, the probabilities for the following criteria are calculated. For the last criterion leave the probabilities of hits and false alarms corresponding to the highest estimate. Based on the values ​​obtained in the usual way, the sensitivity indexes are calculated - d, , C and C ' and the RHP curve is constructed.

Let's explain the logic of calculations on the example of hypothetical data presented in Table. 7.1. In its upper part results of an estimation of probability of hit and false alarms for each value of a five-point scale are reflected. In this case, each response of the subject to a meaningful stimulus is treated as a hit, and on an empty one as a false alarm. Other categories of the answer (correct negations and omissions) are missing in this procedure. In order to evaluate these probabilities, it is necessary to determine the number of answers of each category in the confidence assessment for empty and significant samples and divide the results by the total number of samples of each type.

At the bottom of Table. 7.1 shows the accumulated probabilities of hits and false alarms for each position of the decision criterion C, which are specified by the category of the selected answer.

Table 7.1

Determine the points of the PXP curve based on the results of the confidence assessment

Probability according to confidence assessment

Assurance Rating

1

2

3

4

5

Hits

0.01

0.11

0.33

0.40

0.15

False alarms

0.09

0.34

0.38

0.16

0.03

End of the table. 7.1

Probability according to confidence assessment

Assurance Rating

1

2

3

4

5

C,

C 2

C

C 4

Hits

1.00

0.99

0.88

0.55

0.15

False alarms

1.00

0.91

0.57

0.19

0.03

In order to better understand how the answer probabilities are calculated for each criterion value, imagine that the participant participating in the experiment, whose procedure corresponds to the "yes/no" method, has established a decision criterion equal to the value < strong> C 1 Then the choice of the answer of the category 1 will meet his "no" response, and all other categories will correspond to the "yes" answers. Consequently, the probability of hits in this case will be equal to the sum of probabilities that correspond to the sum of confidence estimates from two to five points. In our example, this amount is 0.99. Similarly, the probability of false alarms is estimated. It turns out to be 0.91.

Now, imagine that our test subject's decision criterion is equal to C 2 . In this case, the probability of his positive answers is obvious , should be equal to the sum of the probabilities of responses corresponding to an estimate of three to five points. As you can see, in such a situation the probability of hits decreased to a value of 0.88, and the probability of false alarms to 0.57.

By the same principle, you can calculate the probability of hits and false alarms for the remaining values ​​of the criterion.

The data obtained in this way can be represented as a curve of the operating characteristic of the receiver (Figure 7.8).

Fig. 7.8. RHP for the data presented in Table. 7.1

Procedure "same/different

Procedure "same/different involves the presentation of two stimuli to the subject (for example, S 1 and S 2 ). These incentives differ by some unobtrusive value. Obviously, two stimuli give four of their pairing combinations: S i -S l , S 2 -S 2 , S 1 -S 2 and S 2 -S 1 . These combinations are presented repeatedly to the subject in a random order. The number of presentation of pairs of stimuli should be quite large. The task of the subject in each sample is to evaluate whether the stimuli of the pair are the same or different. Note that the first two pairs we listed give the answer "the same" and the other two pairs are "different".

This method is convenient because the subject does not need to compare the stimuli according to some specific characteristic, we only need to evaluate the identity or difference of the stimuli. Therefore, this method is recommended to be used only when the characteristics of the stimuli can not be distinguished in explicit form. For example, two very similar beverages can be offered as stimuli to a subject-for example, "Pepsi" and coca cola & quot ;. All that is required of the subject is to report whether he feels the difference between the two samples.

Data processing is reduced to counting the number of right and wrong answers for the same and different & quot ;. In fact, it is no different from what we know by the "yes/no" method. Indeed, if you take the answers the same & quot ;, then the correct answers of this kind will match the hits, and the wrong answers will give false alarms. Based on the data obtained in this way, the working characteristic of the receiver is constructed and the sensitivity indexes are calculated.

Strange procedure

This method is a further development of the previous one. In each sample, the subject performs three or more observations, and two stimuli are used. In one random half of the samples one of these stimuli is presented in all intervals except one, and the second - only in the remaining one interval. In another random half of the samples, the same presentation scheme is used for another stimulus. The order of presentation of an excellent, strange, stimulus is balanced in different tests. The task of the subject is to determine in what interval the different stimulus was presented.

The processing of the results obtained by this method is practically the same as what we already know in relation to the forced choice method. To estimate the sensitivity, the proportions of the correct answers of the subject are calculated - P (c), which are then converted to d 'values ​​by special tables.

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)