External Validity - General Psychological Practice

External validity

The concept of external validity reflects the possibility of generalizing data beyond the limits of specific experimental conditions. The more categories of people and conditions to which the researcher can disseminate the results of his research, the higher the external validity. If the results of the study are only valid for those people on whom they were obtained, and only under the conditions in which they were obtained, then the external validity is low.

The questions that external valuation allows to answer are very important. For example, is it possible to transfer the results of psychological research from animals to humans? Can the norms of mental development, developed by the classics of the developmental psychology about 100 years ago, be applied to modern children? Is it permissible to use the results of the research obtained on psychological students from Moscow for psychological work with a ninety-year-old grandmother from the province?

The answers to these questions generally depend on the set of additional variables controlled in the study, i.e. Those that, in addition to the independent variable, influence the dependent variable and are controlled by the researcher. Thus, the results obtained by student psychologists can be applied to work with a village grandmother if the researcher proves that the results obtained by him do not change depending on the age, lifestyle and level of human education. In other words, if he proves that these additional variables do not affect the relationship between the independent and dependent variables. For this, of course, it is necessary to evaluate the effect of additional variables (in this case they are age, lifestyle, education) on the dependent variable.

If the effect of additional variables is not controlled, then the study has a lower external validity than the study in which these variables are monitored. For example, a study whose results are obtained only on men has a lower external validity than a study conducted on men and women, because it is proved that its results are exactly true for men, but it is not known whether they will be fair for women. A study conducted on men and women of the same age has a lower external validity than a study conducted on men and women over a wide age range, etc. Sex and age in these examples are additional variables that, in the absence of proper control, reduce the external validity of the study.

When planning any experimental research, the question arises of which side variables need to be controlled and how much. As a rule, variables such as sex, age, education and other sociodemographic data are necessarily controlled. More specifically, the above question can be answered only on the basis of the content of the study. If the author knows that there is a certain factor that limits the effect of the cause-effect relationship, he must control the effect of this factor, i.e. control the side variable.

In order to find out whether there are such factors and what they are, first of all it is necessary to very well study the literature on the subject of research and look for in it such objects and conditions that in a certain way influence this subject (dependent variable) and whose presence in the experimental situation can not be avoided. For example, if the researcher found out that many women relieve stress and negative experiences through shopping, but from literature knows that men and women experience troubles in different ways, then he must necessarily control the influence of the sex of the test subjects as an additional variable.

Compared to internal validity, a violation of external validity has less fatal consequences for the findings of the study. Recall that if the internal validity is violated, the researcher can not make any valid conclusions about the cause-effect relationships between the independent and dependent variables. If external validity is violated, it is possible to establish a cause-and-effect relationship, but it is difficult to identify those conditions in case of violation of which this relationship ceases to be performed. As in the case of internal validity, there are no general recipes that allow you to completely secure your research from the violation of external validity, but there are several of the most common threats to external validity that must be checked.

The effect of interaction between selection and experimental impact.

The main threat of external validity are the characteristics of the subjects. The more common features that distinguish the experimental sample from all other people, the lower the external validity. In a scientific language, such a sample is called unrepresentative. In contrast, a representative sample is a sample in which all the psychological characteristics are presented and in the same ratio as in the general population (that is, among those people on whom the results of the study are supposed to be disseminated).

For example, psychological research of people who committed serious crimes, for obvious reasons, is possible only in places of detention. In this case, only those criminals who were caught by the law enforcement bodies will be available to the researcher. Such a sample is not representative for all criminals, since the criminals surveyed are likely to differ from those criminals who for some reason were not caught. Therefore, the dissemination of the results of psychological research from those caught on any criminals will be illegal, for non-caught criminals identified in the study psychological patterns may not work. Such an unreasonable distribution of findings will be a violation of external validity as a result of the interaction of selection and the experimental impact.

In this example, it was a natural restriction of the sample, it is impossible to make a sample of criminals more representative. However, researchers often limit sampling by themselves, neglecting external validity.

Suppose a researcher studies the characteristics of overcoming stress in adolescents with low levels of mental tension living in the Suvorov Cadet Corps, but concludes that the results obtained are valid for all adolescents. Most likely, it is impossible to transfer the data of this study to teenagers living in ordinary families and not thinking about a military career. The sample of cadets is unrepresentative for the study of stress in adolescents, since the cadets are very different from the majority of adolescents in terms of their living conditions, and therefore also by sources of stress, and by the level of mental tension accompanying stress. And if the researcher tries to help ordinary teenagers in the fight against stress, using the knowledge that he received at the Suvorov cadets (that is, to extend this knowledge beyond the sample), his help will most likely not be very effective.

When the subjects of the sample under study differ from the general population, as in these examples, the selection effect takes place. In this case, the experimental impact can be effective only on this sample, but not to lead to the same effect on other people, which differ from the sampled sample. Therefore, this effect is called the interaction effect of selection and the experimental effect.

Any research is subject to this effect, but it poses a particular danger for studies with small and unrepresentative samples. The smaller the sample, the higher the probability that the subjects will be similar for some psychological characteristics. As a result, the causal relationship established in the study will be fair, but only for people with such characteristics, just as in the previous example, the methods of coping with stress that are formed in adolescents are valid only for adolescent cadets.

The same problems arise in sufficiently large samples, if they are unrepresentative. For example, many modern studies involving hundreds of subjects are built on the results obtained by student psychologists. The sample of students has a rather limited representativeness, since students are distinguished from the general population by young age, higher intelligence and learning ability (on the basis of these qualities they are selected to universities), higher prosperity in the family (within five years the student, as a rule, can not self to provide for itself), a higher level of claims and a number of psychological features. Therefore, the results obtained even on sufficiently large samples of students also suffer from violations of external validity. It is unlikely that these results without additional research can be used to work with any representatives of the general population.

All these additional variables are not obvious and, as a rule, are not taken into account by the researchers when formulating the conclusions. Therefore, the description of a sample of subjects acquires such importance in any scientific report. The more detailed the sample, the more understandable its specific features in comparison with the population and the more accurately it is possible to estimate the efficiency of the transfer of results obtained on this sample to people who did not participate in the study. The latter differ from the subjects examined in their personal characteristics, the more efficient the transfer will be.

Another effect that threatens external validity is the choice of experimental conditions that cause such a response of the test subjects, which does not allow the dissemination of the obtained data beyond the limits of a particular experimental situation. The threat of this effect arises when the researcher did not control all the additional variables connected with the organization of the research conditions, and some of them turned out to be important for the investigated cause-and-effect relationship to be manifested. In other circumstances in which such important conditions are absent, the causal relationship will not manifest itself.

One of the most striking examples of this is the possibility of transferring the results of the experiment from the laboratory to the field conditions. Thus, many of the patterns obtained in the laboratory of W. Wundt, not only did not stop working, but lost their relevance, since they were reproduced only within the walls of the laboratory.

As a concrete example we can mention the study by H. Hochmann, devoted to the verification of the assumption, first formulated in the works of W. James, that the subjective experience of emotions is a sensation of bodily reactions to external influences (we are sad because we cry) . Hohmann suggested that if this assumption is true, then people who are deprived of bodily sensation as a result of the disease should experience less intense emotions, because they have less bodily sensations. Moreover, the intensity of emotional experiences should decrease in proportion to the magnitude of the loss of sensitivity. He interviewed patients with lesions of the spinal cord of varying severity and confirmed his hypothesis. Indeed, the more the bodily sensitivity of the respondents was disturbed, the more strongly they changed the emotional experiences they reported. However, repeated studies have shown that this connection is fair, apparently, only for those subjects who changed their lifestyle after the disease (they had to change jobs, give up sports hobbies or hobbies). Those subjects whose lifestyle prior to the illness was mostly sedentary, not related to sports activities, did not report significant differences in the intensity of their experiences before and after the disease compared with the control group of healthy people.

Circumstances that Hohman did not take into account were a change in the way of life of the subjects. In a situation in which such a change occurred, the subjects really lost the intensity of emotional experiences. However, it can not be said that the forced change of lifestyle was the reason for such changes in emotional experience and that it serves as a real source of weakening experiences (ie, a secondary variable). Many people, being healthy, have to change their way of life, but this does not lead to changes in the experience of emotions. By changing the way of life, caused by a serious illness that caused a greater or lesser loss of body sensitivity, appeared to be the unrecognized condition that led to a decrease in the external validity of the study. As a result, the results obtained in it can not be transferred to any patients, but only to those who, like the subjects of the initial experiment, had to change their way of life as a result of the disease.

This effect is due to difficulties in monitoring all components of the research situation (research conditions). It poses a special threat to field research, in which subjects are not protected from a variety of unexpected effects from a large number of additional conditions, as in the laboratory. And a way to control this threat is to provide a detailed description of the sample and test the stability of the causal relationship to the effect of various additional variables.

Reactive effect, or the interaction effect of testing. The essence of this effect is that a person who has experience of passing a pre-test, behaves in an experiment differently and shows other results, compared to a person without such experience. Preliminary testing can set a person to work, make them more sensitive to the experimental impact, tire and lower their efficiency or motivation for research. As a result, the data obtained in such an experiment will be distorted by the fact of preliminary testing, i.e. they will not correspond to those reactions that even the same subjects will give in the absence of preliminary testing.

For example, in order to assess the emotional impact of music, it is necessary to ask the subjects to assess their emotional state by a set of emotional scales before and after listening to the musical fragment. But subjects who passed through preliminary testing and focused on their emotions will take special care in their emotions and when listening to music. Let's say that the results of the research will show that this music has a certain effect on emotions. But is it possible to transfer these results to the general population and say that other people will be affected by this music in a similar emotional way? No, it is impossible, because other people, members of the general population, did not participate in the study and, accordingly, were not subjected to preliminary testing, did not focus on their emotions, and the emotional impact of music did not affect them.

The impact of pre-testing is an additional variable that limits possible situations of using research data. This influence is particularly affected by such studies that use unusual (not similar to the standard survey or test) forms of preliminary (and, respectively, final) testing, when preliminary testing is long, laborious, demotivates (boring, incomprehensible, etc.) creates in the subjects true or false expectations about the experimental impact - in short, increases or decreases the sensitivity to the associated effects.

The method of controlling the effect of pre-testing is Solomon's plan described above when discussing the threat of internal validity.

Mutual interference of experimental effects. This effect is due to the fact that the experience of participation in psychological experiments is accumulating. Subjects who have received various experimental influences do not necessarily lose and forget the result after the experiment is completed. In any case, they still have an idea of ​​the tasks they have solved, and of the experiences caused by the experimental impact. This experience is actualized by the repeated passage of psychological research and distinguishes the results of subjects who participated many times in the studies from the experience of people who did not participate in such studies before.

Dissemination of the findings of the study without taking into account this experience also carries a threat of external validity, since it is not known whether subjects without experience in participating in psychological research will demonstrate the same reactions as experienced subjects. Most likely, the reactions of these subjects will differ, and those cause-effect relationships that are established on some subjects will not work in the case of other experiences that differ from them.

So, a person who has already passed the tests of intelligence, performs them differently than the one who faces them for the first time, and this can lead to differences in the results of these two people. Man is a flexible and trained being that recycles any experience and uses it for more successful further behavior. Participation in psychological experiments is even more such an experience. People who previously participated in the experiments may become more sensitive to the experimental impact or, conversely, acquire the skills to confront it, they understand the instructions to tasks more quickly and quickly, since they have already encountered something like this, more often correctly understand the experimenter, more quickly cope with testing and ask fewer questions.

Such a reaction of the subjects to the experimental impact, conditioned by the experience of multiple participation in research, serves as an additional variable threatening external validity. Without controlling this variable, the results of experienced subjects can not be transferred to a person who has no experience of participating in psychological experiments. To transfer the results of newcomers to people who had experience of participating in psychological research, too, is impossible - this will be a violation of the external validity of the study.

Since it is difficult to find such subjects who have never participated in any surveys in their lives, or have never had a psychological or pseudo-psychological test at least out of curiosity, the interaction effect is not uncommon. The probability of its occurrence is higher in those studies in which a specific experimental effect is used.

Examples of such impact can serve as a demonstration of psychological illusion, the use of emotionally loaded research material, deception of the subject. Thus, for example, in the famous experiments of S. Milgram on the study of conformal behavior, subjects were required to beat an unknown person (an experimenter's assistant) with an electric current, at the request of the researcher. At the conclusion of the experiment, they learned that the researcher was really interested in how soon they would refuse to perform the task and what pressure they needed to exert, so that they would continue acting at the experimenter's pointing. Later, when they became test subjects in some other experiments, it is unlikely that these people treated with the same confidence in instructions and researchers as those who did not get such experience.

To control this effect, you can compare the results of subjects who have more and less experience of participating in psychological research. If there are differences between these groups based on the results of preliminary or final testing, it means that the experience of participation in the research influenced the results of the work of the test subjects and should be taken into account when further using the research data.

In general, the nature of the threat of external validity, the signs of exposure to various threats and actions for their control are collected in Table. 11.3.

Table 11.3

Threats to external validity, features of the research procedure that increase the risk of these threats, and their control

Effect

The threat of external validity (additional variable)

Features of the research procedure that contribute to the appearance of the effect

Control methods

The interaction of selection and experimental impact

Psychological quality, the same for all subjects and distinguishing them from the general population

Low number of subjects

Using a large representative sample

End of the table. 113

Effect

The threat of external validity (additional variable)

Features of the research procedure that contribute to the appearance of the effect

Control methods

Influence of the experimental conditions causing such a reaction of the test subjects, which does not allow to extend the results beyond these conditions

The circumstance unknown to the researcher in the research situation, limiting the manifestations of the causes of the no-effect relationship so that it is present only if it exists

The vast majority of field studies (see the chapter "Psychologist's Functional Responsibilities"), in which it is impossible to take into account all the impacting conditions

There are no common methods, it is necessary to minimize the effects on the subjects to those that can be taken into account and monitored

Reactive effect, or test interaction effect

The experience of passing the initial testing creates differences in the susceptibility to the experimental impact and distinguishes the subjects from all who do not have it, i.e. from the general population

All measurements that create a specific experience that increases the sensitivity to experimental effects

Solomon's plan

Mutual interference of experimental effects

Experience in participating in psychological research, creating differences in susceptibility to experimental effects and behavior in the study

All procedures that create specific experiences, including the use of unusual tools

Comparison of the results of experienced subjects with the results of those. who had not previously participated in the research

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)