Determination of the reliability of the measuring instrument...

Determining the reliability of the measuring tool

The accuracy, objectivity of any psychological dimension depends on how the methodology is worked out, how correctly the tasks are selected from the point of view of their mutual consistency, how homogeneous it is. The internal homogeneity of the methodology shows that its tasks actualize the same property, attribute.

To check the reliability of the measuring tool, which speaks of its homogeneity (or homogeneity), the so-called "splitting" method is used. Usually tasks are divided into even and odd, separately processed, and then the results of the two series obtained are correlated with each other. It is possible to divide tasks into the methods related to the first half and to the second half of the methodology. To apply this method, you need to place the subjects in such conditions that they can manage to solve (or try to solve) all tasks. If the methodology is uniform, then there will not be much difference in the success of the solution for such halves, and therefore the correlation coefficient will be high enough.

You can divide tasks in another way. For example, you can match the first and third quarter with the second and fourth, etc. However, the splitting on even and odd jobs seems most appropriate, because this is the most independent way from the influence of factors such as training, training, fatigue, etc.

The technique is considered reliable when the coefficient is not less than 0.75. The best reliability tests give coefficients of the order of 0.90 or more.

Determining the stability of the trait under study

Determining the reliability of the methodology itself does not mean solving all the issues related to its application. We still need to establish how stable, stable is the sign that the researcher intends to measure. It would be a methodological mistake to count on the absolute stability of psychological signs. The fact that the measured sign changes with time, there is nothing dangerous for reliability. The whole point is, to what extent do the results vary from experience to experiment in one and the same subject, do not these fluctuations lead to the fact that for some unknown reason the subject turns out to be in the beginning, in the middle, or at the end of the sample. It is impossible to draw any concrete conclusions about the level of representation of the measured trait in such a subject. Thus, fluctuations in the trait should not be unpredictable. If the reasons for which there is a sharp fluctuation are not clear, then such a sign can not be used for diagnostic purposes.

To test the stability of the feature to be diagnosed, a technique known as test-retest is used. It consists in re-examining the subjects using the same technique. The stability of the trait is judged by the correlation coefficient between the results of the first and second surveys. It will indicate the preservation or non-retention of each participant in the sequence number in the sample.

A variety of factors affect the degree of stability and stability of a diagnosed property. The number of them is quite large. We have already discussed how important it is to comply with the uniformity of the procedure for the experiment. So, if the first testing was conducted in the morning hours, then the second one should be conducted in the morning; if the first experiment was accompanied by preliminary demonstration of tasks, then when this test is repeated this condition must also be observed, etc.

In determining the stability of the trait, the time interval between the first and second examination is of great importance. The shorter the period from the first to the second test, the more (with other things being equal) the chances that the diagnosed trait will keep the level of the first test. With the increase in the time interval, the stability of the trait tends to decrease, since the number of extraneous factors affecting it increases. Therefore, the conclusion suggests that it is advisable to conduct a second diagnosis in a short time after the first. However, there are complications: if the time between the first and second experience is small, some subjects can reproduce their previous responses from memory and, thus, depart from the meaning of the tasks. In this case, the results of the two presentations of the methodology can no longer be regarded as independent.

It is difficult to clearly answer the question of how long it can be considered optimal for a repeated experiment. Only the researcher, based on the psychological essence of the methodology, the conditions in which it is conducted, the characteristics of the sample of subjects, should determine this period. At the same time, such a choice should be scientifically justified. In the test literature, the most frequent are the time intervals of several months (but no more than six months). When examining young children, when age changes and development occur very quickly, these intervals can be on the order of several weeks.

The stability coefficient of the methodology should be sufficiently high (not lower than 0.80).

Definition of constancy

Ie. relative independence of the results from the personality of the experimenter. Since the technique developed for diagnostic purposes is not intended to remain forever in the hands of its creators, it is very important to know to what extent its results are influenced by the personality of the experimenter. Although the diagnostic technique is always supplied with detailed instructions for its application, rules and examples that indicate how to conduct the experiment, but regulate the behavior of the experimenter, the speed of his speech, tone of voice, pause, facial expression is very difficult. The subject in his attitude to the experience will always reflect how the experimenter himself refers to this experience (assumes negligence or acts exactly in accordance with the requirements of the procedure, shows exactingness, perseverance or lack of control, etc.).

Although in the practice of practice, the criterion of constancy is used infrequently, but this can not serve as a basis for its underestimation. If the authors of the method have suspicions about the possible influence of the personality of the experimenter on the outcome of the diagnostic procedure, it is advisable to check the methodology by this criterion. It is important to keep in mind the following. If, under the influence of a new experimenter, all the subjects began to work a little better or a little worse, then this fact (although it deserves attention) to the reliability of the technique will not have an effect. Reliability will change only when the influence of the experimenter on the subjects is different: some articles work better, others are worse, and others are just like the first experimenter. In other words, if the subjects under the new experimenter changed their ordinal positions in the sample.

The coefficient of constancy is determined by correlating the results of two experiments conducted under relatively identical conditions on the same sample of subjects, but by different experimenters. The correlation coefficient should not be below 0.80.

So, three indicators of reliability of psychodiagnostic techniques were considered. A question may arise whether it is necessary to check each of them when creating the methodologies. This question should be answered in the affirmative. So the American authors of the APA (American Psychological Association) in the book "Standard requirements for pedagogical and psychological tests" in the Reliability note that the coefficient of reliability is a generic concept that includes several species, and each species has its own special meaning. This point of view is shared by KM Gurevich. In his opinion, when they talk about different ways of determining reliability, they are not dealing with a better or worse measure, but with measures of different in essence reliability. In fact, what is the procedure, if it is not clear whether it is reliable in itself as a measuring tool, or is the stability of the measured property not established? What is the diagnostic technique, if it is not known, can the results vary depending on who is conducting the experiment? Each individual indicator can not replace other methods of verification and, therefore, can not be considered as a necessary and sufficient characteristic of reliability. Only a technique with a complete reliability characteristic is most suitable for diagnostic and practical applications.

thematic pictures

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)