Lecture 09: ANOVA Recap and Factorial Designs

Situating ANOVA Within the GLM

What is the General Linear Model (GLM)?

• GLM is a framework that describes a broad set of statistical models.
• Includes techniques like:
  • ANOVA (Analysis of Variance)
  • Multiple Regression
  • ANCOVA (Analysis of Covariance)
  • MANOVA (Multivariate ANOVA)

What is the General Linear Model (GLM)?

• Unified by the equation:

\[ Y = X\beta + \epsilon \]

Visualizing the General Linear Model (GLM)

Figure 1: Graphical Representation of the General Linear Model with Intercept(5) and Slope(2).

Key Elements of the GLM Equation 1/2

1. Outcome Variable (Y):
   • The dependent variable being predicted or explained.
   • Usually continuous in both regression and ANOVA models.
2. Predictors (X):
   • Independent variables in regression (e.g., continuous variables).
   • Grouping variables in ANOVA (e.g., categorical variables).

Key Elements of the GLM Equation 1/2

3. Coefficients (β):
   • Represent the estimated weights or effects of predictors (X) on the outcome (Y).
4. Error Term (ε):
   • Captures the variation in Y that is not explained by the predictors.

ANOVA as a Special Case of GLM

• ANOVA compares means across groups by partitioning variance:
  • Total Variance = Variance Between Groups + Variance Within Groups
• In GLM terms:
  • X: Encodes group membership using dummy variables.
  • β: Represents group mean effects.
  • ε: Residual error capturing unexplained variance.

Extending to Factorial Designs: Main Effects

• Factorial designs include multiple predictors (e.g., A and B).
• Each predictor contributes to the outcome variable (Y) through:
  • Main Effects:
    • Effect of A (e.g., differences across levels of factor A).
    • Effect of B (e.g., differences across levels of factor B).
  • Residual Error: Variation unexplained by predictors.

Extending to Factorial Designs: Interaction Effects

• Interaction effects occur when the effect of one predictor (A) depends on the level of the other predictor (B).
• Combined model for a factorial design: \[ Y = \beta_0 + \beta_1 X_A + \beta_2 X_B + \beta_3 (X_A \times X_B) + \epsilon \]
• Partitioned Variance in Factorial ANOVA:
  • Main effects of A and B.
  • Interaction effect (A × B).
  • Residual error. ## ANOVA vs. Multiple Regression

How They Are Related

• ANOVA:
  • Focuses on group differences (categorical predictors).
  • Example: Comparing means of 3 groups.
• Multiple Regression:
  • Allows continuous and categorical predictors.
  • Example: Predicting an outcome using test scores and group membership.
• Connection:
  • ANOVA is a regression model with categorical predictors encoded as dummy variables.

Example: One-Way ANOVA as Regression

• Regression model for a one-way ANOVA:

Y = β₀ + β₁X₁ + β₂X₂ + ε

Components:

• X₁, X₂: Dummy variables for groups.
• β₁: Difference between group 1 and the reference group.
• β₂: Difference between group 2 and the reference group.

Visualizing the Relationship

Figure 2: ANOVA as regression: Group means represented by dummy variables.

Key Takeaways

GLM Framework

• ANOVA and Regression are special cases of GLM.
• Both share the same foundation:
• Y = Xβ + ε:
  • Represents the General Linear Model (GLM).
  • (Y): Outcome variable (dependent variable).
  • (X): Predictors (independent variables).
  • (β): Coefficients for predictors.
  • (ε): Residual error term.

Practical Differences

• ANOVA emphasizes group comparisons.
• Regression allows for both continuous and categorical predictors.

Recap on Factorial Designs

What Are Factorial Designs?

• Factorial designs involve multiple independent variables (IVs).
• Common in psychology research for testing complex interactions.
• Goal: To interpret main effects and interactions.

Why Focus on 2x2 Designs this year?

• Simplest factorial design with:
  • 2 Independent Variables (IVs).
  • Each IV has 2 levels.
• Patterns in data can show:
  • Main Effects: Influence of each IV independently.
  • Interactions: Combined influence of IVs.

Understanding the 8 Possible Patterns

Figure 3: Bar graphs: 8 possible outcomes in 2x2 design.

Exploring Main Effects and Interactions

Interpreting Main Effects

• Main effects represent the consistent influence of an IV.
• Example: Coffee and wakefulness:
  • More coffee = Higher alertness, regardless of location.

Interaction Effects

• Interaction occurs when one IV’s effect depends on the level of another.
• Example: Coffee may affect wakefulness differently in the morning vs. at night.

Best Practices for Factorial Designs

• Keep designs simple: Fewer IVs and levels make interpretation easier.
• Ensure sufficient data for all combinations of IV levels.
• Always visualize data before analysis to understand patterns.

Summary

• 2x2 designs are foundational in understanding factorial experiments.
• Interactions complicate main effects but reveal critical relationships.
• Use clear visualizations to enhance interpretation.

Citation and Acknowledgments

Research Methods Lecture 09