Department of Psychology

Non-experimental design:

In a non-experimental design, no treatments are given. Because there are no treatments, a non-experimental design can’t predict causal relationships. Instead, this design studies naturally occurring variation in the independent and dependent variables without any intervention (by the researcher or anyone else) to equate cases prior to their exposure to the independent variable. This design is used when using an experimental design is either impossible or unethical. The following are examples of non-experimental designs:

Survey: Describes attitudes beliefs, and behaviors of a population. A survey of some members of the population is called a sample, a survey of the entire population is called a census. Surveys can also be used in quasi- and true-experimental designs.

Case study: An intensive study of one person

Ethnographic research: An intensive case study of a group.

Historical research: An examination of data to understand the past

Quasi-experimental design:

The difference between a quasi-experimental and true experimental design is that a quasi-experimental design does not use random assignment of participants. However, the groups vary in critical ways such that they are different from each other. You would use a design like this when you can’t control what category the independent variable falls under, or when it would be unfeasible or unethical to do so. (For example, studying the effects of violent television on juvenile delinquents vs. non-delinquents—you can’t assign juveniles to one of the two groups). There are two major types: Nonequivalent control group designs and Before-and-after designs.

Non-equivalent control group designs have experimental and control groups that are designated before the treatment occurs and are not created by random assignment.

Before-and-after designs have a pretest and posttest but no comparison group—the subject essentially serves as its own control based on pretest behavior.

True experimental design:

A true experimental design is one that uses completely random assignment (R) of groups. The design can be demonstrated graphically. In the examples below, “R” means random assignment, O means observation, X means treatment. Each line represents a group.

Pretest-posttest design
R      O     X     O      
R      O             O

This is an example of a pretest-posttest design. This way, you can measure how effective the treatment (X) is by looking at the difference between the pre- and post-tests for the control and experimental groups. However, this design is subject to sensitization. The fact that the experimental group is taking a pre-test may influence its performance for the treatment.

Post-test only
R        X     O
R                O

This is a post-test only randomized control group design. The advantage of this design is that there won’t be any pre-test sensitization. However, because there’s no pre-treatment data, it’s nearly impossible to gauge the effect of the treatment.

The Solomon Four-Group Design
R      O       X       O
R      O                 O                                                             R                X        O                                                                                 R      O

This is a combination of the above two designs. This way, the tester can see what is gained and the effect of prior testing. However, because it doubles the number of groups needed, it is more expensive and requires twice the number of participants. Ideally, it would always be used. However, given other limitations, it is generally used when control of pre-test and post-test and treatment effects is absolutely necessary.

Factorial design:

This method can be used when you’re interested in looking at an independent variable with more than one level. For example, you could be looking at the effect of anti-depressants vs. therapy in young and old people. Your design would look like this (the numbers are the difference between pre-test and post-test “Happiness scale”—for example, the “young” group averaged a 9-point increase between tests using anti-depressant therapy; they’re more happy after the treatment than before it):

This is a 2 x 2 factorial design. The numbers correspond to how many levels, or categories, of each variable are being tested. (i.e. if we were to test teenagers, working adults, and retired people, it would be a 2 x 3 factorial design)

You could also have a design with three variables instead of two; if each variable has two levels it would be a 2 x 2 x 2.

Using this method, both levels of the independent and dependent variables are tested, meaning that every combination of every category within each variable is examined. The following questions can be answered:

Which treatment is more effective?

Which group recovers faster?

Are different treatments effective for different ages?

To find which treatment is more effective, average the columns and see which is greater. In this example, the average improvement for anti-depressants is (9+4)/2, or 6.5. The average improvement for therapy is 8.5.

To find which group recovers faster, average the rows and see which is greater. In this example, the average improvement for young people is 7.5. The average improvement for old people is also 7.5

These are known as main effects: they answer in general which treatment was most effective.

However, there is also an interaction effect: that the different levels of the IV and DV interact to give different results for different groups. If you were to graph this data, you would find that the lines would cross.

A two-way ANOVA is one inferential statistic that examines interaction effects. There are also way three-way, four-way, etc ANOVAs to examine more complex factorial designs.

Within-Groups designs:

A within-groups design means that instead of comparing two groups, a control and an experimental, you compare the group’s before and after results—a participant acts as his own control. However, if administering multiple treatments, the study must be carefully designed to avoid sequence effects. Sequence effects can occur when a participant receives multiple treatments or conditions. Sometimes, results can be due to the order of the treatment, not because of the treatment itself. For example, if a researcher is testing the effectiveness of several anti-psychotic medications on schizophrenics, and she wants to use the same patients for each medication, she may find that all of the drugs work really well. In reality, it could be that the first drug worked really well, and the other ones don’t make much of a difference, but because the first drug worked so well, its effects lasted and made the other drugs appear to work too.

There are three ways sequence effects can be controlled:

Cross-over design:
R   O    X   O         O
R   O          O   X   O

Counter-balanced (for two groups):
A     B     C     D
D     C     B     A

Latin four-square (for four groups):
A   B    C   D
B   C    D   A                                                                                              C   D    A    B                                                                                             D   A    B    C

Back to Research and Design

References:

Patten, Mildred L. (2002). Understanding research methods: An Overview of the essentials (3rd ed.). Los Angeles: Pyrczak Publishing.

Pyrczak, Fred. (2002). Success at statistics: A Worktext with humor (2nd ed.). Los Angeles: Pyrczak Publishing.

Schutt, Russell K. (1999). Investigating the social world: the Process and practice of research (2nd ed.). Thousand Oaks: Pine Forge Press.