1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

Can factors be significant if there is a lack of fit and weak R²?

Discussion in 'DOE - Design of Experiments' started by Rockychano, May 20, 2019.

  1. Rockychano

    Rockychano New Member

    Joined:
    May 20, 2019
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Hello everybody,

    after my ANOVA the R² of the model is really bad (R² = 0,1743) and it has a significant lack of fit (lack of fit = 0,0096). I used 8 factors but only one was selected as significant.

    1.) The lack of fit tells me, that the model needs to be extended by another term (quadratic term, 2-way-interaction or a completely new term). Is that right?

    2.) One factor was identified as highly significant (p = 0,012). Is it ok to trust this result, although the model (lack of fit, R²) is really bad?

    Thank you
     
  2. Miner

    Miner Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    576
    Likes Received:
    492
    Trophy Points:
    62
    Location:
    Greater Milwaukee USA
    Welcome to QFO.

    Your assessment is essentially correct. Your one factor that was identified as significant probably is significant. The model, however, is not a good model. As you speculated, there are a number of reasons for this such as missing factors, interactions, quadratic terms, or a covariate. Also look at your data to see whether you have a few data points with large residuals. Investigate and validate these as correct. Adding the correct missing factors should improve your R². If you address the lack of fit and find that you still have a poor R², you then need to investigate the sources of your experimental error starting with the measurement system.
     
    Rockychano and Bev D like this.
  3. Bev D

    Bev D Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    605
    Likes Received:
    663
    Trophy Points:
    92
    Location:
    Maine
    What Miner said. Plus remember that the R squared value is essentially a measure of the amount variation due to the factor(s) being evaluated. In other words a low R square means that most of the variation is driven by one or more factors that were present in the study but not tested for. Also a very important thing that is too often overlooked is that statististical significance (no matter how low the p value is) does not mean practical importance. In other words the p value does not tel you if the identified factor is the dominant or controlling factor. To have a useful model you must have identified the controlling and occasionally strongly contributing factors.
     
    Rockychano and Miner like this.
  4. Miner

    Miner Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    576
    Likes Received:
    492
    Trophy Points:
    62
    Location:
    Greater Milwaukee USA
    Some experiments can best be described per the following quote by Andrew Gelman:

    "My best analogy is that they are trying to use a bathroom scale to weigh a feather—and the feather is resting loosely in the pouch of a kangaroo that is vigorously jumping up and down." Kangaroo.png
     
    Rockychano, Bev D and Andy Nichols like this.