Reliability of experimental research - General Psychological Workshop

Reliability of experimental research

A great influence on the formation of ideas about the rules for the correct construction of experiments in the social sciences was provided by the work of the American researcher D. Campbell. He defined four criteria for a good experiment. These criteria generalize what we have already noted in the first paragraph of this chapter.

1. Clarification of the temporal sequence between the alleged cause and effect. The conclusion that some event is the cause, and the other is the consequence, of the worst cases in the event that no third events intrude between cause and effect. If a third event occurs between the alleged cause and effect, this raises doubts: whether the investigation has changed under the effect of the alleged cause or whether the third event occurred. For example, the causal relationship between a kick and a dog bite is easier to prove when the dog bites a man immediately after a kick than three days during which this particular dog had many more contacts with that same person.

2. Establishment of a joint change (covariance) of the alleged cause and effect. In other words, the researcher must prove that changing the alleged cause always leads to a change in the effect, and the intended consequence does not change independently, without changing its causes. If the alleged cause and effect change only together with one another (covariates), then this can be considered a reliable sign of their connection. In the above example, with the influence of stuffiness on the success of solving problems, cause and effect are covariant, if the subjects always solve problems better in ventilated rooms and worse - in stuffy ones. If, however, when the proposed cause changes, the effect does not always change, but only in some cases, then the relationship between them should be questioned (for example, if the subjects sometimes solve problems worse in the stuffy rooms than in the ventilated ones, and sometimes not worse). The same applies to the situation where the assumed cause is unchanged, and the implied consequence changes by itself (for example, subjects in ventilated rooms solve problems better, then worse than in stuffy ones). In this case, the existence of links between the alleged cause and effect also raises doubts.

3. Excluding the influence of side effects that could explain the relationship between cause and effect. This was discussed in more detail above, when discussing the signs of a cause-effect relationship. The covariance of the success of solving problems and the state of air in the room can be explained not only by the cause and effect relationship, but also by other influences - the connection of the competitor's motivation, the influence of fatigue, poisoning in the dining room between the experiment stages and much more.

4. Elimination of alternative hypotheses about constructs included in the causal relationship. Two sources of such alternative hypotheses are possible: the existence of third factors linking cause and effect (this was discussed in the previous paragraph), and the existence of certain relationships, other than cause and effect, between the constructs included in the causal relationship. In a good experiment, the event, which is defined as the alleged cause, and the alleged consequence are not connected by any other connections, except the alleged cause-and-effect. Such additional relationships can be the ratio of part and whole, source and derivative, hierarchical subordination, the relationship of partial coincidence of individual components in the structure of the alleged cause and effect. Such relationships form alternative explanations for the relationship between the alleged cause and effect. These alternative explanations in general form can be formulated as follows: the relationship between the two events is not causal, but, for example, hierarchical.

To clarify the last condition put forward by Campbell, consider this example. Suppose the researcher tests the hypothesis that the use of effective coping strategies leads to a reduction in mental stress and stress. The problem is that coping strategies are a definition of the actions a person takes to combat stress and psychological tension. It turns out that the experimental impact (the use of effective coping strategies) includes the notion of overcoming stress and mental stress, t.p. what the researcher is only going to fix. If, as a result of the subject's actions, they do not decrease, it means that he uses an inefficient coping strategy, and something else. Therefore, in this study, one can not infer the causal relationship between the independent and dependent variables, because these variables are otherwise related - as a source and the effect they produce.

The procedures for planning the experiment are aimed at ensuring that the research procedure developed by the researcher meets the listed criteria as closely as possible. These procedures will be discussed later. They provide an improvement in the quality of experimental work, which is determined using the concepts of reliability and validity of the experiment.

First, let's talk about the reliability of the study. It reflects the number of errors made during the planning of the study. The more such errors, the lower the reliability. The study is considered reliable if its results are well reproduced in different circumstances, by different researchers and at different times. The fact that the results are reproduced well under the new circumstances means that all the necessary experimental conditions are fixed, the methods of measuring variables rarely give errors, all the necessary variables are controlled.

If in the morning the results of the experiment are the same, in the evening others, and on the weekends - the third, the results of the Moscow test subjects differ from the results of the St. Petersburg, then such results can not be called reliable. The same goes for reproducing the results by different researchers. If one researcher gets the results that confirm the hypothesis of the experiment, the other one - refutes it, and in the third the results confirm the contiguity, then this experiment can not be considered reliable.

The reliability of the research results is checked not only by repeating it. Another way is to compare the results of a new study with the results of already known and proven studies. If these results are in good agreement, then this is an indication of the reliability of the new study.

However, in some cases, if a new area is being studied in which little research has been done, and they are very fragmented, or when a new study is inconsistent with the results of earlier ones and proven reliable, the researcher will have to repeat his experiment more than once prove that it is reliable.

The most impressive example of this is the study of the phenomenon of taste aversion. In the experiments on the development of conditioned reflex connections in rats, J. Garcia (subsequently a professor emeritus at the University of California, holder of many titles and awards) revealed an interesting fact. When the rats received food and were exposed to strong radioactive radiation, they felt very ill as a result of irradiation, they felt sick, and they very quickly - from one or two combinations - learned to no longer touch the food they were being irradiated during the reception.

The essential difference between this experiment and the classical Pavlovian experiments was that the effect of irradiation affects not immediately, but after a while (in the case of rats, after about half an hour). For these half an hour, rats were exposed to a wide variety of influences, which, however, were much closer to being disturbed by the time, but nevertheless, the only stimulus that the rats began to avoid was food, during which they were irradiated.

This result contradicts the main postulate, formulated as a result of studies of classical conditioning, that there should not be any other effects between the conditioned stimulus (food) and the unconditioned (poor state of health as a result of irradiation) that will weaken their connection; conditional and unconditional stimulus

Should be served together. Nevertheless, in these experiments, the relationship between food taste and nausea was formed much faster than in the results of the classical procedure for the formation of conditioned reflexes.

Garcia was difficult to publish his first results, solid psychological journals refused his articles, but he continued experiments, instead of radioactive radiation, he began using a similarly-acting lithium chloride. Following Garcia, these experiments were conducted by other scientists. In addition to rats, studies were conducted on other species of animals, including non-laboratory animals such as wild crows and coyotes, and similar results were obtained each time. As a result, the scientific community was forced to recognize the rightness of the researcher, he proved that his results are highly reliable. Garcia's works have become widely known, and the very phenomenon of a particularly rapid formation of conditional connections between more "suitable" each other with stimuli (such as food and nausea) was called the Garcia effect.

This researcher had to prove the reliability of his results for a long time, carrying out many repeated experiments in different conditions, with different experimenters and experimental animals, with different influencing substances causing nausea, because his results did not agree, and in some places directly contradicted the results of numerous experiments formation of conditioned reflexes. ІІО he proved that his results are not accidental, are not caused by the error of the technique, the influence of the experimenter or the characteristics of the experimental animals. Only after that his data received a well-deserved recognition.

In order to be able to independently and to enable others to assess the reliability of their research, the researcher should as accurately and fully describe the procedure for the study: the order of the tasks, the instructions to the subjects, the questions of the subjects and their answers to them, important for the study of the condition. This provides an opportunity for anyone to accurately repeat the research, transfer it to other samples and to other conditions without distorting the essence of the experiment.

The results of such a re-examination will be as much as possible comparable to the results of the initial one to assess its reliability. If the researcher describes the procedure carelessly or does not describe it at all, then he does not give the opportunity to check the reliability of his research, the procedure of which can not be repeated by no one, since only his author remembers instructions, tasks and influences on subjects that led to such results as were obtained in his research.

thematic pictures

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)