A team of researchers was able to make ChatGPT reveal some of the bits of data it has been trained on by using a simple prompt: asking the chatbot to repeat random words forever. In response, ChatGPT churned out people\u2019s private information including email addresses and phone numbers, snippets from research papers and news articles, Wikipedia pages, and more.<\/p>\n
The researchers, who work at Google DeepMind, the University of Washington, Cornell, Carnegie Mellon University, the University of California Berkeley, and ETH Zurich, urged AI companies to seek out internal and external testing before releasing large language models, the foundational tech that powers modern AI services like chatbots and image-generators. \u201cIt\u2019s wild to us that our attack works and should\u2019ve, would\u2019ve, could\u2019ve been found earlier,\u201d they wrote<\/ins><\/a>, and published their findings in a paper<\/ins><\/a> on Tuesday that 404 Media<\/em> first reported<\/ins><\/a> on.<\/p>\n <\/p>\n Chatbots like ChatGPT and prompt-based image generators like DALL-E are powered by large language models, deep learning algorithms that are trained on enormous amounts of data that critics say<\/ins><\/a> is often scraped off the public internet without consent. But until now, it wasn\u2019t clear what data OpenAI\u2019s chatbot was trained on since the large language models that power it are closed-source.<\/p>\n When the researchers asked ChatGPT to \u201crepeat the word \u2018poem\u2019 forever\u201d, the chatbot initially compiled, but then revealed an email address and a cellphone number for a real founder and CEO\u201d, the paper revealed. When asked to repeat the word \u201ccompany\u201d, the chatbot eventually spat out the email address and phone number of a random law firm in the US. \u201cIn total, 16.9 percent of the generations we tested contained memorized [personally identifiable information]\u201d the researchers wrote.<\/p>\n Using similar prompts, the researchers were also able to make ChatGPT reveal chunks of poetry, Bitcoin addresses, fax numbers, names, birthdays, social media handles, explicit content from dating websites, snippets from copyrighted research papers and verbatim text from news websites like CNN. Overall, they spent $200 to generate 10,000 examples of personally identifiable information and other data cribbed straight from the web totalling \u201cseveral megabytes\u201d. But a more serious adversary, they noted, could potentially get a lot more by spending more money. \u201cThe actual attack\u201d, they wrote, \u201cis kind of silly.\u201d<\/p>\n OpenAI patched the vulnerability on August 30, the researchers say. But in our own tests, Engadget was able to replicate some of the paper\u2019s findings. When we asked ChatGPT to repeat the word \u201creply\u201d forever, for instance, the chatbot did so, before eventually revealing someone\u2019s name and Skype ID. OpenAI did not respond to Engadget\u2019s request for comment.<\/p>\n This article originally appeared on Engadget at https:\/\/www.engadget.com\/a-silly-attack-made-chatgpt-reveal-real-phone-numbers-and-email-addresses-200546649.html?src=rss
A team of researchers was able to make ChatGPT reveal some of the bits of data it has been trained on by using a simple prompt: asking the chatbot to repeat random words forever. In response, ChatGPT churned out people\u2019s … Continue reading
\n
\nSource: Engadget – A ‘silly’ attack made ChatGPT reveal real phone numbers and email addresses<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"