Recently, Adobe released a free AI-powered audio processing tool that can improve some low-quality audio recordings by removing background noise and making the sound more powerful. When it works, the result sounds like a recording made in a professional sound booth with a high-quality microphone.
The new tool, called Enhance Speech, appeared as part of an AI research project called Project Shasta. Recently, Adobe renamed Project Shasta to Adobe Podcast.
Enhance Speech is free to use but requires an Adobe account to be created and works best in a desktop web browser. After signing up, users can upload an MP3 or WAV file up to an hour long or 1GB in size. After a few minutes you can listen to the result in your browser or download the resulting cleaned audio.
In our tests with the service, Enhance Speech worked best with audio with no crosstalk or excessive noise. For example, we recorded the voice of a person standing 3 meters away from an iMac’s built-in microphone, including nearby fan noise, and the resulting sound (once processed by Enhance Speech) sounded like it was recorded closely without noise. Professional microphone studio.
How does it work? Adobe didn’t provide any details, but we suspect the company is training a deep learning model on many (possibly thousands) hours of clean and loud sound. The model can then “learn” to select human voice frequencies and synthesize a facsimile that exactly matches the source. This is speculation and we’ve reached out to the company for comment until Adobe provides more technical details.
In this regard, some Hacker News commentators have reported hallucinatory results (unexpected outputs such as ghostly voices where the AI misinterprets the input sound) from extremely loud sounds (such as speech recorded near a waterfall) or from non-English sources. Enhance Speech does more than just a traditional noise reduction technique.
Enhance Speech is not the first tool to provide such AI-powered noise reduction capability. For example, an open source package called Mayavose and a commercial service called Audo Studio do something similar.
It’s worth noting that Enhance Speech is part of a larger suite of AI-powered podcasting tools from Adobe, including a Mic Check tool (also available for free now) and a transcript-based audio editing tool still in the invite-only process. . Beta testing.