By M. Louis Gordon
The principle guiding our workflow in recording and mixing tends to be to separate everything. The more individual elements of a mix are recorded cleanly on individual tracks, the more control we have over our product. When sounds are “married” to each other in a recording, it is often the painstaking work of separating those pieces — through spectral repair, EQ-ing or otherwise — that precedes any sort of mix stage.
Well, what if you could just plop a mic in front of a band in a noisy setting, hit record, and manipulate each instrument, voice, and background detail all from that one track? That’s the idea behind the software Hit’n’Mix Infinity, and it does a heck of a good job at its mission. By importing a Wav. file of a full music mix into the program’s “ripping” algorithm, it breaks the track up into parts, displaying each voice as a unique color on a timeline, reminiscent of a beefed-up MIDI piano roll.
Percussion, noise and other atonal sounds are put onto high-register and low-register spectrograms above and below the timeline. You see a strikingly visual representation of your mix, with a suite of tools to cut, erase, draw and manipulate all of these “notes” of sound.
Hit’n’Mix targets post sound as much as it is to music production. It is equipped with a special setting to rip vocals/SFX, a de-noising processor and a greenscreen option to overlay video behind the timeline, allowing one to edit sync-sound to video. It seems hard to imagine a program this fresh on the scene contending with some of the hefty power tools that have ripened with us in our mix rooms, but I was intrigued by the possibilities Infinity could open up.
Identifying and isolating voices within a single recording could, for instance, address a perennial issue those in post face: overlapping dialog. Clever editing and some spectral repair can sometimes yield an acceptable remedy for such a problem, but all too often, expensive ADR sessions are needed to fully eliminate it.
A similar and quite mortifying situation is the case of headphone bleed; when the sound engineer’s voice, through a stray headset in the recording booth, is picked up alongside an actor or vocalist’s voice. It’s a human error that is sometimes discovered only when recording has finished and the talent has left the building. It was with this latter predicament that I decided to put Infinity to the test.
I chose a work sample that would challenge some of the drawbacks I’ve seen in Infinity’s ripping algorithm; it has much more success isolating smooth, tonal vowels than it does consonants. I input a recording of an actor laughing and speaking on-mic in the booth, with prominent headphone bleed of the engineer speaking and overlapping her. I set Infinity to rip the file as a “Vocal/FX” source.
The initial rip wasn’t too shabby. Infinity clearly translates sound level into color opacity on the MIDI piano roll, so I could very quickly identify the talent’s on-mic voice (bold yellow) from the softer headphone bleed (dim yellow). With a few swift drag-selects and deletes, I was able to remove a good chunk of the bleed voice.
As good as Infinity’s algorithm may be, it is not always able to distinguish between voices to the level that the human brain can. When the actor’s voice ran into the engineer’s voice at a similar pitch, the two were grouped into the same “note.” I then needed to pick through some of these notes and cut them apart manually with the “split” tool.
Many “notes” were also really combinations of both voices as they overlapped. Infinity’s “Note Editor” was the utility to turn to for these issues.
Visually, it is a little hard to identify what exactly you’re dealing with. The gist of the Note Editor is essentially a blown-up, blocky spectrogram of a selected note in which you can raise, lower, smooth or erase pieces of the spectrum, among more complex operations. Though it was hard to navigate, I was struck with the ease with which I could isolate very specific tonal elements and remove them. It even rivaled my experience doing such operations with the industry stalwart for spectral repair, iZotope RX.
But it was after this point in the process that I reached the Achilles heel of the program. After the work of splitting, editing and removing individual “notes,” much of the headphone bleed still remained. I could clearly hear the engineer’s voice in much of the recording, missing only the clearest midrange of the original. Most of that sound data of the headphone bleed was ripped into the “unpitched” ranges at the top and bottom of the piano roll, including lots of vocal sounds that were clearly tonal.
Switching into “Edit Unpitched” mode on the toolbar showed me what I still needed to deal with.
Strewn throughout the unpitched audio were pieces of the engineer’s voice. So, much like one would do in iZotope RX, I went through the spectrogram selecting and playing sounds, determining what was the talent’s voice and what was the engineer’s voice. Unlike RX, however, there doesn’t seem to be much capability with adjusting the spectrogram color and contrast. Softer noises are hardly visible in this view without a well-calibrated picture monitor, and it took a good deal of hunting and pecking to spot each unwanted detail. But after a little patience, I had managed to remove all the headphone bleed from the recording.
The process, though far from turnkey, did yield some impressive results. I was able to isolate the talent’s voice with minimal artifacting* (see note at end). A full sentence spoken by the engineer under a long, nasal breath from the talent was completely gone, leaving the desired voice in pretty good detail. I was struck with how much control and precision Infinity lends to what is often regarded as a sound editor’s nightmare.
At the same time, there is certainly room for improvement from Hit’n’Mix. Removing all that headphone bleed took with it much of the noise floor. The result sounded as if it were hard-gated; with the air sucked out of the recording and the transients of the actor’s voice lacking tails. These are rough edges that a little room tone and light reverb could alleviate, but that is one more step in the repair process we often don’t have time for.
Final Thoughts
Hit’n’Mix Infinity wants to wear a lot of different hats; from MIDI editor to spectral repair to pitch-shifter to sampler. It can seem as if its attention is a little divided. With such a Swiss-army knife of tools, there ought to be a more efficient workflow to remove an unwanted voice that doesn’t involve editing operations in three different territories (notes, Note Editor, Unpitched sounds). These modes, particularly the Note Editor and Unpitched spectrogram, could use a makeover to aid in the editing process.
The performance of the ripping algorithm also has some room to improve, especially with non-musical material. While it can usually identify different instruments with ease, speech from different speakers is a different story. One speaker’s word would be grouped together with another in the same note, with the bass plosives and sibilance split up in the low and high “unpitched” ranges. A lot of time was spent correcting for the errors made by the rip. I frequently had to account for elements of a speaker’s voice showing up in a tonal note and in the spectrogram. This made voice manipulation a complicated affair.
Perhaps further development on a unique “spoken word” ripping mode tuned to address these issues would help Hit’n’Mix make headway in the world of post production. For now, it may not be worth the time to clean up a bad recording if a vocalist can get back in the booth and run another take. When such an option would be costlier than an hour or two of sound editing work, however, Infinity proves itself a powerful tool to get the job done.
* I should note that preserving the desired vocal detail takes some care. Harmonics in the human voice can sometimes get split up into multiple notes and unpitched ranges in the ripping process. Artifacting can appear very easily if some of these elements are accidentally erased.
[Editor’s Note: We reached out to Hit’n’Mix to ask if they are continuing to work on features for audio post. They said they are and expect what they call “a significant upgrade release” in the next few months. Improvements will include a workflow boost to the Note Editor and Unpitched spectrogram.]
Louis Gordon is sound engineer for Silver Sound, and film producer/director based in Boston. He has contributed to mixes for Vice News Tonight, The CW’s Tough Mudder: The Challenge Within, and 2016 Sundance Official Selections Equity and Goat.
