{"id":117899,"date":"2023-11-20T14:24:00","date_gmt":"2023-11-20T18:24:00","guid":{"rendered":"https:\/\/www.shortform.com\/blog\/?p=117899"},"modified":"2023-11-22T10:59:55","modified_gmt":"2023-11-22T14:59:55","slug":"observation-selection-effect","status":"publish","type":"post","link":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/","title":{"rendered":"The Observation Selection Effect: A Handy Tool for Bullshitters"},"content":{"rendered":"\n<p>How much damage can improper data collection create? How can manipulators leverage it for their own advantage?<\/p>\n\n\n\n<p>In <em>Calling Bullshit<\/em>, Carl T. Bergstrom and Jevin D. West contend that bullshit often arises when data-based arguments rely on flawed data. They explain how the observation selection effect is an example of this and show how some people take advantage of it to deliberately deceive others.<\/p>\n\n\n\n<p>Continue reading to learn about the observation selection effect and how it can wreak havoc.<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-the-observation-selection-effect\">The Observation Selection Effect<\/h2>\n\n\n\n<p>Bergstrom and West maintain that <strong><a href=\"https:\/\/www.shortform.com\/blog\/selection-bias-in-research\/\">selection bias<\/a> can creep into data via the <em>observation selection effect<\/em><\/strong>, which occurs when our data collection process is correlated with the variable that we\u2019re collecting data on. For example, if we could conduct presidential polls only via smartphones, the correlation between smartphone ownership and presidential preference would undermine the integrity of these polls\u2014smartphone owners may be wealthier, on average, and therefore more likely to vote for a candidate who cuts taxes for the rich.<\/p>\n\n\n\n<p>(Shortform note: The observation selection effect causes notorious issues in philosophy and science, where the fact that we exist to observe certain phenomena <a href=\"https:\/\/ndpr.nd.edu\/reviews\/anthropic-bias-observation-selection-effects-in-science-and-philosophy\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>presupposes <\/em>certain conditions about the universe<\/a>. For example, some philosophers take the observation that the universe is surprisingly hospitable to sentient life as evidence of a creator, even though this hospitability is <em>necessary <\/em>for us to observe it in the first place.)&nbsp;<\/p>\n\n\n\n<p>The observation selection effect, Bergstrom and West argue, can yield misleading conclusions that bullshitters can prop up. Returning to the previous example, bullshitters could use poll data conducted exclusively by smartphone to spread a <a href=\"https:\/\/www.shortform.com\/blog\/false-narratives\/\">false narrative<\/a> that one candidate was <em>guaranteed <\/em>to win, when in fact their chances might be much lower than the poll suggests.<\/p>\n\n\n\n<p>(Shortform note: In the case of polling, one reason bullshitters might want to use biased polls is that such polls can influence elections via the contagion effect\u2014the <a href=\"https:\/\/www.press.umich.edu\/pdf\/0472099213-ch11.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">tendency of voters to rally around candidates who are polling well<\/a>. For example, if a misleading poll suggested that one candidate was crushing the other candidates in a primary election, that could cause more voters to view that candidate\u2019s victory as inevitable and therefore vote for them.)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>How much damage can improper data collection create? How can manipulators leverage it for their own advantage? In Calling Bullshit, Carl T. Bergstrom and Jevin D. West contend that bullshit often arises when data-based arguments rely on flawed data. They explain how the observation selection effect is an example of this and show how some people take advantage of it to deliberately deceive others. Continue reading to learn about the observation selection effect and how it can wreak havoc.<\/p>\n","protected":false},"author":9,"featured_media":38477,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[34,25],"tags":[1331],"class_list":["post-117899","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-communication","category-statistics","tag-calling-bullshit","","tg-column-two"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v24.3 (Yoast SEO v24.3) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The Observation Selection Effect: A Handy Tool for Bullshitters - Shortform Books<\/title>\n<meta name=\"description\" content=\"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here&#039;s how.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Observation Selection Effect: A Handy Tool for Bullshitters\" \/>\n<meta property=\"og:description\" content=\"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here&#039;s how.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\" \/>\n<meta property=\"og:site_name\" content=\"Shortform Books\" \/>\n<meta property=\"article:published_time\" content=\"2023-11-20T18:24:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-22T14:59:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/s3.amazonaws.com\/wordpress.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"640\" \/>\n\t<meta property=\"og:image:height\" content=\"424\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Elizabeth Whitworth\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Elizabeth Whitworth\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\"},\"author\":{\"name\":\"Elizabeth Whitworth\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/d2928cf6c11a69ced1491d6a5b74fb13\"},\"headline\":\"The Observation Selection Effect: A Handy Tool for Bullshitters\",\"datePublished\":\"2023-11-20T18:24:00+00:00\",\"dateModified\":\"2023-11-22T14:59:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\"},\"wordCount\":372,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg\",\"keywords\":[\"Calling Bullshit\"],\"articleSection\":[\"Communication\",\"Statistics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\",\"url\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\",\"name\":\"The Observation Selection Effect: A Handy Tool for Bullshitters - Shortform Books\",\"isPartOf\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg\",\"datePublished\":\"2023-11-20T18:24:00+00:00\",\"dateModified\":\"2023-11-22T14:59:55+00:00\",\"description\":\"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here's how.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage\",\"url\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg\",\"contentUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg\",\"width\":640,\"height\":424},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.shortform.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Observation Selection Effect: A Handy Tool for Bullshitters\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#website\",\"url\":\"https:\/\/www.shortform.com\/blog\/\",\"name\":\"Shortform Books\",\"description\":\"The World&#039;s Best Book Summaries\",\"publisher\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.shortform.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#organization\",\"name\":\"Shortform Books\",\"url\":\"https:\/\/www.shortform.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png\",\"contentUrl\":\"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png\",\"width\":500,\"height\":74,\"caption\":\"Shortform Books\"},\"image\":{\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/d2928cf6c11a69ced1491d6a5b74fb13\",\"name\":\"Elizabeth Whitworth\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1fff9d65a52ac4340660218e7b63ee5e365cf08e7aa7adff79a0142cd4b96f84?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1fff9d65a52ac4340660218e7b63ee5e365cf08e7aa7adff79a0142cd4b96f84?s=96&d=mm&r=g\",\"caption\":\"Elizabeth Whitworth\"},\"description\":\"Elizabeth has a lifelong love of books. She devours nonfiction, especially in the areas of history, theology, and philosophy. A switch to audiobooks has kindled her enjoyment of well-narrated fiction, particularly Victorian and early 20th-century works. She appreciates idea-driven books\u2014and a classic murder mystery now and then. Elizabeth has a Substack and is writing a book about what the Bible says about death and hell.\",\"sameAs\":[\"rina@shortform.com\"],\"award\":[\"Contributions to joint task force efforts (FBI)\",\"Contributions to Special Operations Division (DOJ & DEA)\",\"Efforts to fight the war on drugs (NSA)\",\"Contributions to Operation Storm Front (US Customs Service)\"],\"knowsAbout\":[\"History\",\"Theology\",\"Government\"],\"jobTitle\":\"Senior SEO Writer\",\"worksFor\":\"Shortform\",\"url\":\"https:\/\/www.shortform.com\/blog\/author\/elizabeth\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"The Observation Selection Effect: A Handy Tool for Bullshitters - Shortform Books","description":"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here's how.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/","og_locale":"en_US","og_type":"article","og_title":"The Observation Selection Effect: A Handy Tool for Bullshitters","og_description":"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here's how.","og_url":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/","og_site_name":"Shortform Books","article_published_time":"2023-11-20T18:24:00+00:00","article_modified_time":"2023-11-22T14:59:55+00:00","og_image":[{"width":640,"height":424,"url":"https:\/\/s3.amazonaws.com\/wordpress.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","type":"image\/jpeg"}],"author":"Elizabeth Whitworth","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Elizabeth Whitworth","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#article","isPartOf":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/"},"author":{"name":"Elizabeth Whitworth","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/d2928cf6c11a69ced1491d6a5b74fb13"},"headline":"The Observation Selection Effect: A Handy Tool for Bullshitters","datePublished":"2023-11-20T18:24:00+00:00","dateModified":"2023-11-22T14:59:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/"},"wordCount":372,"commentCount":0,"publisher":{"@id":"https:\/\/www.shortform.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage"},"thumbnailUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","keywords":["Calling Bullshit"],"articleSection":["Communication","Statistics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/","url":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/","name":"The Observation Selection Effect: A Handy Tool for Bullshitters - Shortform Books","isPartOf":{"@id":"https:\/\/www.shortform.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage"},"thumbnailUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","datePublished":"2023-11-20T18:24:00+00:00","dateModified":"2023-11-22T14:59:55+00:00","description":"The observation selection effect can yield misleading conclusions that bullshitters can prop up. Here's how.","breadcrumb":{"@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.shortform.com\/blog\/observation-selection-effect\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#primaryimage","url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","contentUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","width":640,"height":424},{"@type":"BreadcrumbList","@id":"https:\/\/www.shortform.com\/blog\/observation-selection-effect\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.shortform.com\/blog\/"},{"@type":"ListItem","position":2,"name":"The Observation Selection Effect: A Handy Tool for Bullshitters"}]},{"@type":"WebSite","@id":"https:\/\/www.shortform.com\/blog\/#website","url":"https:\/\/www.shortform.com\/blog\/","name":"Shortform Books","description":"The World&#039;s Best Book Summaries","publisher":{"@id":"https:\/\/www.shortform.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.shortform.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.shortform.com\/blog\/#organization","name":"Shortform Books","url":"https:\/\/www.shortform.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png","contentUrl":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2019\/06\/logo-equilateral-with-text-no-bg.png","width":500,"height":74,"caption":"Shortform Books"},"image":{"@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/d2928cf6c11a69ced1491d6a5b74fb13","name":"Elizabeth Whitworth","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.shortform.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1fff9d65a52ac4340660218e7b63ee5e365cf08e7aa7adff79a0142cd4b96f84?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1fff9d65a52ac4340660218e7b63ee5e365cf08e7aa7adff79a0142cd4b96f84?s=96&d=mm&r=g","caption":"Elizabeth Whitworth"},"description":"Elizabeth has a lifelong love of books. She devours nonfiction, especially in the areas of history, theology, and philosophy. A switch to audiobooks has kindled her enjoyment of well-narrated fiction, particularly Victorian and early 20th-century works. She appreciates idea-driven books\u2014and a classic murder mystery now and then. Elizabeth has a Substack and is writing a book about what the Bible says about death and hell.","sameAs":["rina@shortform.com"],"award":["Contributions to joint task force efforts (FBI)","Contributions to Special Operations Division (DOJ & DEA)","Efforts to fight the war on drugs (NSA)","Contributions to Operation Storm Front (US Customs Service)"],"knowsAbout":["History","Theology","Government"],"jobTitle":"Senior SEO Writer","worksFor":"Shortform","url":"https:\/\/www.shortform.com\/blog\/author\/elizabeth\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/www.shortform.com\/blog\/wp-content\/uploads\/2021\/06\/ux-indonesia-8mikJ83LmSQ-unsplash.jpg","_links":{"self":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/117899","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/comments?post=117899"}],"version-history":[{"count":7,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/117899\/revisions"}],"predecessor-version":[{"id":117972,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/posts\/117899\/revisions\/117972"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/media\/38477"}],"wp:attachment":[{"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/media?parent=117899"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/categories?post=117899"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shortform.com\/blog\/wp-json\/wp\/v2\/tags?post=117899"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}