{"id":118957,"date":"2023-12-07T11:33:00","date_gmt":"2023-12-07T15:33:00","guid":{"rendered":"https:\/\/www.shortform.com\/blog\/?p=118957"},"modified":"2023-12-12T11:34:11","modified_gmt":"2023-12-12T15:34:11","slug":"ai-model-collapse","status":"publish","type":"post","link":"https:\/\/www.shortform.com\/blog\/ai-model-collapse\/","title":{"rendered":"AI Model Collapse: What It Is & the Risks It Poses"},"content":{"rendered":"\n

What is AI model collapse? What are the consequences of model collapse? Is it already happening?<\/p>\n\n\n\n

Model collapse is the complete erosion of AI models that occurs when they\u2019re trained on flawed content generated by other AI models. Model collapse can lead to nonsensical, inaccurate, and biased AI output. Experts say that diverse, human-curated data is key to stopping AI model collapse.<\/p>\n\n\n\n

Keep reading to learn about AI model collapse, and for examples of it happening today.<\/p>\n\n\n\n\n\n\n\n

AI Could Destroy Itself<\/h2>\n\n\n\n

AI \u201cmodel collapse\u201d\u2014where AI models rapidly deteriorate due to being trained on flawed\u00a0content generated by other AI models\u2014poses serious risks. But can this runaway train be stopped?<\/p>\n\n\n\n

Background<\/strong><\/h3>\n\n\n\n

Artificial Intelligence (AI) models are computer programs that perform tasks such as language translation or image recognition<\/a> by analyzing vast amounts of information, known as datasets. They identify patterns within this data to make predictions and enhance their performance and accuracy over time.<\/p>\n\n\n\n

However, AI systems can experience “model collapse<\/a>,” a problem where they\u2019re trained on data produced by other AI models without human-created data. This leads to a rapid decline in output quality due to a feedback loop.<\/p>\n\n\n\n

A recent study<\/a> from the University of Cambridge and the University of Edinburgh reveals how each iteration of synthetic training data (artificially generated data) significantly amplifies errors<\/a>: Mistakes replicate like a digital “genetic mutation,” escalating inaccuracies exponentially. The problem is exacerbated by the tendency of AI models to quickly forget<\/a> their original training data. Consequently, AI systems can completely collapse after just seven to 10 generations, producing nothing but gibberish.<\/p>\n\n\n\n

Model Collapse Implications<\/strong><\/h3>\n\n\n\n

This significant breakdown in AI function isn’t just theoretical; it’s evident in real-world applications such as AI-generated news articles. <\/strong>NewsGuard, a company that assesses news platform credibility, has identified more than 500 news outlets<\/a> that primarily rely on AI to generate news articles with minimal human oversight. This lack of meaningful human involvement in the process and recursive reliance on AI can lead to a cascade of compounded errors, resulting in news stories being consistently marred with inaccuracies.<\/p>\n\n\n\n

A case in point<\/a> is Microsoft’s MSN news portal, which relies heavily on AI to generate stories without human editorial supervision. In September, it published an article describing deceased NBA player Brandon Hunter as \u201cuseless at 42<\/a>.”\u00a0\u00a0<\/p>\n\n\n\n

While the presence of inappropriate or incorrect content in AI-generated news articles is concerning, experts say the potential amplification of harmful biases due to model collapse presents an even more serious risk.<\/p>\n\n\n\n

Looking Ahead<\/strong><\/h3>\n\n\n\n

Experts say human creativity is key to preventing collapse<\/strong><\/a>. Currently, engineers scrutinize data to prevent AI from self-training on its synthetic output. <\/p>\n\n\n\n

Given the potential threats related to model collapse\u2014including an emerging cybersecurity risk\u2014experts advise AI companies and researchers to treat training data as a threat. <\/strong>They recommend that those working with the technology take proactive measures to prevent it from self-destruction, including:<\/p>\n\n\n\n