AI’s Role in Creating Harmful Images of Indian Women is a Dangerous Trend.

The marriage season has started and social media is flooded with so many photos and videos of people getting married. Recently, a photo came to our notice where an Indian bride was standing with the groom wearing a yellow bikini along with a Dupatta and jewellery. Sharing the photo on Facebook, users wrote that the bride was from Lucknow, and she broke the stereotype by wearing a Banarasi bikini at her marriage ceremony.

Facebook Link | Archived Link.

The image has been widely circulated on social media. But did the bride really wear the Bikini at her wedding? 

By conducting a reverse image search, we found the viral image on Reddit uploaded by a user on Sub-Reddit “DesiAdultFusion” on 18 November 2024. There were several other similar images with other different brides.

The Sub-Reddit had a description that read, “DesiAdultfusion is a subreddit dedicated to showcasing AI-generated artwork related to desi culture. From traditional to contemporary, our community welcomes all forms of desi-inspired art created using artificial intelligence. Share your unique creations and explore the work of others who share your passion for all things desi. Join us in celebrating the beauty and diversity of desi culture through the lens of AI art.”

An AI detector tool also detected the image as AI created.

While going through the user’s profile on the Sub-Reddit we came across various adult images, created by AI, uploaded there. 

Over the recent years, Artificial Intelligence has become a significant part of the evolving technology. Artificial Intelligence is benefiting mankind by reducing human error, saving time, making unbiased decisions, automating repetitive tasks, increasing operational efficiency, processing large volumes of data and enhancing customer experiences. 

But, AI has also reduced employment, and it has privacy and data security concerns. It can be programmed to harm, it can diminish critical thinking abilities.

One of the biggest risks of AI in social media is the Deepfake.  A deepfake is an edited video, audio or photo that is created using artificial intelligence. Artificial intelligence is used to replace a person’s appearance or voice with another or to artificially create a person’s voice to show that the person is doing or saying something that he or she does not do. Deepfake also refers to computer-generated content that does not exist in real life.

Some users on social media platforms like Reddit, Facebook, Instagram, etc, are using AI-generated images as satirical content, which eventually goes viral as a real one. This practice is increasing the risks of spreading social media misinformation. It has been argued that AI will become so powerful that humanity may irreversibly lose control of it. 

According to a report published in The Business Standard on 27 July 2024, “The Ministry of Electronics and IT (MeitY) is drafting a new law focused on artificial intelligence (AI), which will notably avoid prescribing penal consequences for violations, recognising the technology’s significant benefits, according to a report by The Indian Express. This legislation will be a standalone law, which will require social media platforms such as Facebook, Instagram, YouTube, and X to include watermarks and labels on AI-generated content.

MeitY is also exploring legal frameworks to mandate companies developing large language models to train their systems on Indian languages and context-specific content.

The discussion around AI content warnings has been prominent since last year. In India, it gained attention after deepfake videos of actors and citizens surfaced and AI systems like Google’s Gemini showed inconsistencies in responses about political figures, including Prime Minister Narendra Modi.”

Please read here.

Artificial Intelligence needs to be used ethically. Rather than becoming a slave of AI, we, human should focus on using AI for the upliftment of mankind.