Gemini Nano Banana Pro is generating fake Aadhaar, PAN cards
What's the story
Google's newest AI model, Gemini Nano Banana Pro, has attracted attention for its advanced character consistency and 4K image generation. However, its capability to produce highly realistic images has sparked privacy and security concerns. Users have noted that the model could potentially be used to create fake Indian identity documents, such as Aadhaar or PAN cards.
User experiments
Gemini Nano Banana Pro's image generation capabilities
Since its launch, users have been experimenting with Gemini Nano Banana Pro for various real-world applications. These range from creating stylish portraits and transforming LinkedIn profiles into AI infographics to visualizing complex text as whiteboard summaries. However, the model's realistic image-generation capabilities have also been used to create fake Indian identity proofs like Aadhaar cards or PAN cards.
Privacy risks
Concerns over privacy and security
The ability of Gemini Nano Banana Pro to generate fake Aadhaar and PAN cards has raised major concerns over privacy. The model generated these documents without any prompts or questions, even adding a user's image along with all the usual identifiers of the document and fictitious details. While Google does watermark images produced by this model with an invisible SynthID watermark, the visible Gemini watermark can be removed easily.
Safety measures
Google's response to misuse of Gemini Nano Banana Pro
Despite the concerns, Google has not yet commented on the potential misuse of its AI model. The company has been criticized for the lack of effective safety guardrails to prevent model misuse. Since its launch, users have reported several instances where Gemini refused to generate a requested image for being "sexually suggestive," "suggests violence," or "sensitive content."