{"id":72134,"date":"2022-07-21T11:32:00","date_gmt":"2022-07-21T15:32:00","guid":{"rendered":"https:\/\/www.shortform.com\/blog\/?p=72134"},"modified":"2022-08-01T12:08:04","modified_gmt":"2022-08-01T16:08:04","slug":"what-does-statistically-significant-mean","status":"publish","type":"post","link":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/","title":{"rendered":"What Does “Statistically Significant” Mean?"},"content":{"rendered":"\n

What is significance in statistics? What does “statistically significant” mean?<\/p>\n\n\n\n

In statistics, significance refers to the degree of confidence or certainty in the results of a statistical analysis being attributable to a specific cause. When results are said to be “statistically significant,” the observed relationship between variables is unlikely to be due to chance. <\/p>\n\n\n\n

Keep reading to learn about the concept of statistical significance, explained in simple terms. <\/p>\n\n\n\n\n\n\n\n

Statistical Significance Explained<\/h2>\n\n\n\n

What does “statistically significant” mean? The results of statistical analyses are often reported as being \u201cstatistically significant\u201d or \u201cnot statistically significant.\u201d When results are statistically significant, it means that you can be reasonably certain that your observed results are due to the variable you are measuring and not to random chance.<\/strong><\/p>\n\n\n\n

Statistical significance is often reported with a statistic called the p-value. The p-value pinpoints how likely your results are given a true null hypothesis. In other words, it tells you the likelihood of getting your results if the variable you\u2019re measuring really has no effect and chance alone influenced the data. <\/p>\n\n\n\n

A small p-value adds confidence to rejecting the null hypothesis. <\/strong>When the p-value is less than your established confidence level, you can report your results as statistically significant<\/em>. <\/p>\n\n\n\n

For example, say our vitamin company chooses a .05 confidence level for their statistical analysis, meaning that they want to be 95% confident when they reject the null hypothesis and move forward with the new formula. Their statistical analysis shows a p-value of .01. This means that there is a 1% chance that their results are due to random chance, and the null hypothesis is true (in other words, there is a 99% chance that these results are not<\/em> just due to chance). Since this p-value falls within their 95% confidence level for rejecting the null hypothesis, they report their results as \u201cstatistically significant.\u201d <\/p>\n\n\n\n

P-Value Misconceptions<\/strong><\/h3>\n\n\n\n

The p-value is a tool for determining whether the values in our dataset are \u201csignificant\u201d from a statistical perspective. Since the p-value can be a tricky value to grasp, people have several common misconceptions about it<\/a>. We\u2019ll highlight two of them.<\/p>\n\n\n\n

First, people mistakenly believe the p-value communicates how strong an effect one variable has on another. For example, if your data showed a statistically significant relationship between eating spinach and being able to lift heavier weights an hour later, and your p-value was .001, you might infer that spinach has a large impact on strength. However, a small p-value only means a small likelihood that your results were due to random chance. The reality might be that participants were able to lift just an extra two ounces after eating spinach.<\/p>\n\n\n\n

The second common misconception is that a small p-value means that you are likely to get the same result if you run the experiment again. In our above example, for instance, since your p-value of .001 indicates a .1% chance that your results were due to random chance, you might infer that you don\u2019t need to do the experiment again because your results are pretty conclusive. However, the p-value is specific to the dataset being analyzed and can vary widely even between different samples testing the same variable. This is especially true with smaller sample sizes.<\/p>\n","protected":false},"excerpt":{"rendered":"

What is significance in statistics? What does “statistically significant” mean? In statistics, significance refers to the degree of confidence or certainty in the results of a statistical analysis being attributable to a specific cause. When results are said to be “statistically significant,” the observed relationship between variables is unlikely to be due to chance. Keep reading to learn about the concept of statistical significance, explained in simple terms.<\/p>\n","protected":false},"author":7,"featured_media":70628,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[25],"tags":[681],"class_list":["post-72134","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-statistics","tag-naked-statistics","","tg-column-two"],"yoast_head":"\nWhat Does "Statistically Significant" Mean? - Shortform Books<\/title>\n<meta name=\"description\" content=\"What does the expression "statistically significant" actually mean? Here's what significance tells us about data, explained in simple terms.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What Does "Statistically Significant" Mean?\" \/>\n<meta property=\"og:description\" content=\"What does the expression "statistically significant" actually mean? Here's what significance tells us about data, explained in simple terms.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\" \/>\n<meta property=\"og:site_name\" content=\"Shortform Books\" \/>\n<meta property=\"article:published_time\" content=\"2022-07-21T15:32:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-08-01T16:08:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/s3.amazonaws.com\/wordpress.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Darya Sinusoid\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Darya Sinusoid\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\"},\"author\":{\"name\":\"Darya Sinusoid\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/0421cce75bc249b11e2517b3a91f9c46\"},\"headline\":\"What Does “Statistically Significant” Mean?\",\"datePublished\":\"2022-07-21T15:32:00+00:00\",\"dateModified\":\"2022-08-01T16:08:04+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\"},\"wordCount\":536,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg\",\"keywords\":[\"Naked Statistics\"],\"articleSection\":[\"Statistics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\",\"url\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\",\"name\":\"What Does \\\"Statistically Significant\\\" Mean? - Shortform Books\",\"isPartOf\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg\",\"datePublished\":\"2022-07-21T15:32:00+00:00\",\"dateModified\":\"2022-08-01T16:08:04+00:00\",\"description\":\"What does the expression \\\"statistically significant\\\" actually mean? Here's what significance tells us about data, explained in simple terms.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage\",\"url\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg\",\"contentUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg\",\"width\":1200,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.shortform.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"What Does “Statistically Significant” Mean?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#website\",\"url\":\"https:\/\/www.shortform.com\/blog\/\",\"name\":\"Shortform Books\",\"description\":\"The World's Best Book Summaries\",\"publisher\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.shortform.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\",\"name\":\"Shortform Books\",\"url\":\"https:\/\/www.shortform.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png\",\"contentUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png\",\"width\":500,\"height\":74,\"caption\":\"Shortform Books\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/0421cce75bc249b11e2517b3a91f9c46\",\"name\":\"Darya Sinusoid\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/07\/Untitled-design-1.png\",\"contentUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/07\/Untitled-design-1.png\",\"caption\":\"Darya Sinusoid\"},\"description\":\"Darya\u2019s love for reading started with fantasy novels (The LOTR trilogy is still her all-time-favorite). Growing up, however, she found herself transitioning to non-fiction, psychological, and self-help books. She has a degree in Psychology and a deep passion for the subject. She likes reading research-informed books that distill the workings of the human brain\/mind\/consciousness and thinking of ways to apply the insights to her own life. Some of her favorites include Thinking, Fast and Slow, How We Decide, and The Wisdom of the Enneagram.\",\"url\":\"https:\/\/www.shortform.com\/blog\/author\/darya\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"What Does \"Statistically Significant\" Mean? - Shortform Books","description":"What does the expression \"statistically significant\" actually mean? Here's what significance tells us about data, explained in simple terms.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/","og_locale":"en_US","og_type":"article","og_title":"What Does \"Statistically Significant\" Mean?","og_description":"What does the expression \"statistically significant\" actually mean? Here's what significance tells us about data, explained in simple terms.","og_url":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/","og_site_name":"Shortform Books","article_published_time":"2022-07-21T15:32:00+00:00","article_modified_time":"2022-08-01T16:08:04+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/s3.amazonaws.com\/wordpress.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","type":"image\/jpeg"}],"author":"Darya Sinusoid","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Darya Sinusoid","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#article","isPartOf":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/"},"author":{"name":"Darya Sinusoid","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/0421cce75bc249b11e2517b3a91f9c46"},"headline":"What Does “Statistically Significant” Mean?","datePublished":"2022-07-21T15:32:00+00:00","dateModified":"2022-08-01T16:08:04+00:00","mainEntityOfPage":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/"},"wordCount":536,"commentCount":0,"publisher":{"@id":"https:\/\/www.shortform.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage"},"thumbnailUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","keywords":["Naked Statistics"],"articleSection":["Statistics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/","url":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/","name":"What Does \"Statistically Significant\" Mean? - Shortform Books","isPartOf":{"@id":"https:\/\/www.shortform.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage"},"thumbnailUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","datePublished":"2022-07-21T15:32:00+00:00","dateModified":"2022-08-01T16:08:04+00:00","description":"What does the expression \"statistically significant\" actually mean? Here's what significance tells us about data, explained in simple terms.","breadcrumb":{"@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#primaryimage","url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","contentUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","width":1200,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/www.shortform.com\/blog\/what-does-statistically-significant-mean\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.shortform.com\/blog\/"},{"@type":"ListItem","position":2,"name":"What Does “Statistically Significant” Mean?"}]},{"@type":"WebSite","@id":"https:\/\/www.shortform.com\/blog\/#website","url":"https:\/\/www.shortform.com\/blog\/","name":"Shortform Books","description":"The World's Best Book Summaries","publisher":{"@id":"https:\/\/www.shortform.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.shortform.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.shortform.com\/blog\/#organization","name":"Shortform Books","url":"https:\/\/www.shortform.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png","contentUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png","width":500,"height":74,"caption":"Shortform Books"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/0421cce75bc249b11e2517b3a91f9c46","name":"Darya Sinusoid","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/07\/Untitled-design-1.png","contentUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/07\/Untitled-design-1.png","caption":"Darya Sinusoid"},"description":"Darya\u2019s love for reading started with fantasy novels (The LOTR trilogy is still her all-time-favorite). Growing up, however, she found herself transitioning to non-fiction, psychological, and self-help books. She has a degree in Psychology and a deep passion for the subject. She likes reading research-informed books that distill the workings of the human brain\/mind\/consciousness and thinking of ways to apply the insights to her own life. Some of her favorites include Thinking, Fast and Slow, How We Decide, and The Wisdom of the Enneagram.","url":"https:\/\/www.shortform.com\/blog\/author\/darya\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2022\/07\/calculator-money-budget.jpg","_links":{"self":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/72134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/comments?post=72134"}],"version-history":[{"count":7,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/72134\/revisions"}],"predecessor-version":[{"id":74053,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/72134\/revisions\/74053"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/media\/70628"}],"wp:attachment":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/media?parent=72134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/categories?post=72134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/tags?post=72134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}