1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice
You must be a registered member in order to post messages and view/download attached files in this forum.
Click here to register.

The higher the significance the higher the effect size?

Discussion in 'DOE - Design of Experiments' started by Marcel, Apr 25, 2019.

  1. Marcel

    Marcel New Member

    Joined:
    Mar 19, 2019
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Hello everyone,

    I have a question whether significance (high p-value) and effect size are proportional?

    I did already read that highly significant results do not necessarily need to have a big (practical) effect on my response.

    Definition from Wikipedia: Statistical Significance:
    Effect size
    Main article: Effect size
    Effect size is a measure of a study's practical significance.A statistically significant result may have a weak effect. To gauge the research significance of their result, researchers are encouraged to always report an effect size along with p-values. An effect size measure quantifies the strength of an effect, such as the distance between two means in units of standard deviation (cf. Cohen's d), the correlation coefficient between two variables or its square, and other measures.

    But what if Im doing a design and I have several significant parameters. Not all of them need to be practical significant (effect size) but is it correct to say that as higher the significance is the higher is the effect size (where it does not matter how high the effect size is)?

    Thanks in advance
    Marcel
     
    Last edited by a moderator: Apr 25, 2019
    Marcel, Apr 25, 2019
    #1
  2. Bev D

    Bev D Moderator Staff Member

    Joined:
    Jul 30, 2015
    Messages:
    561
    Likes Received:
    625
    Trophy Points:
    92
    Location:
    Maine
    No. No... no. NO

    The smaller the p value the higher the statistical significance. BUT the effect size is NOT related to the p value.
    The p value is dominated by the within subgroup standard deviation NOT the difference in the subgroup means.

    As Deming and Ott said: plot your data, look at your data and THINK about your data...forget black box statistics.
     
    Bev D, Apr 25, 2019
    #2
    Marcel and Miner like this.
  3. Marcel

    Marcel New Member

    Joined:
    Mar 19, 2019
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Ok thanks, in my case, the lower p-value had the bigger effect but I wondered, if you can generalize that. Thanks a lot!
     
    Marcel, Apr 26, 2019
    #3