by - The Lensa app creates face-changing effects using machine learning and user-uploaded photos.
- Some users received nude pictures of themselves thanks to AI-generated edits.
- The company says Lensa can be tricked into making nudes, but some users say they don’t upload NSFW images.
The trending Lensa app – currently the top photography app in the Apple and Google Play stores – creates artistic edits based on user-uploaded reference photos, but machine learning technology seems to create unintentional nude photos of its users.
One user wrote, “Okay, instead of the first 20 selfies I could find, I put my 20 hottest photos on the lens and came back with a bunch of AI-generated nudes.” Wrote from Twitter. “To be clear, NONE of the photos I posted contained nudity, which the app specifically forbids!”
This sentiment was echoed by dozens of others, most of whom were women, and he said the app automatically creates sexualized or fully nude photos, despite avoiding reference photos that are unsafe for work in their uploads.
Andrey Usoltsev, CEO and co-founder of Lensa’s parent company Prisma Lab, told TechCrunch that while such images “cannot be accidentally generated” by the app, they may be encouraged to create nude images through “deliberate abuse.” terms of service (prohibits uploading “obscene, pornographic, indecent, indecent, obscene” or otherwise sexual content).
While it’s unclear how often the app creates nude images without asking, multiple users are reporting that this applies to them.
“The weird thing is that I never sent any nude photos as this would be against the Lensa app policy, but it still started producing nude photos???” posted by another user from twitter.
Of particular concern for some users is whether the app somehow accesses unloaded photos from its internal storage, and whether the app’s privacy policy allows the use of app-generated data by third-party companies such as Google Cloud Platform and Amazon Web Services. .
“Lensa users: Did you get a highly sexualized image in your avatar pack?” a problem user Wrote from Twitter. “I received a topless, fully frontal nude picture in my package and I’m worried. I’m worried about the rights of this picture and whether the app is accessing other pictures on my phone.”
Usoltsev told TechCrunch that the technology used to create photo edits is increasingly learning and – despite having some content moderation apps – can still be overwhelmed by users or act in unpredictable ways, resulting in the output of bare edits.
“We state that the product is not intended for minors and we warn users of potential content. We also avoid using such images in our promotional materials,” Usoltsev told TechCrunch. “To improve Lensa’s work, we are in the process of creating the NSFW filter. This filter will effectively blur any images detected in this way. Opening or saving such images is at the sole discretion of the user.”
Prisma Labs representatives did not immediately respond to Insider’s request for comment.
