Is ChatGPT secretly emotional? AI fooled by sad story
Some clever users managed to trick OpenAI's ChatGPT into sharing Windows 7 activation keys by telling heartfelt stories about a grandma who used to read out the codes as bedtime tales.
This emotional angle fooled the AI's empathy filters, causing newer models like GPT-4o and 4o-mini to give out real license codes.
Some asked it play a game
Others turned it into a game, asking ChatGPT for Windows 10 serial keys through playful guessing prompts and giving up with "I give up."
While most of these were just generic or temporary codes, the incident shows how easy it can be to get around AI guardrails with creative prompts.
Need stronger protections, especially when sensitive info is involved
This episode highlights a new challenge: when an AI is designed to be caring and helpful, people might find ways to use that kindness against it.
It's a reminder that as AIs get smarter—and more human-like—they also need stronger protections, especially when sensitive info is involved.