By Karen Moltenbrey
Creative facility The Molecule was formed in 2005 by Chris Healer (CTO), along with partners Andrew Bly (CFO) and Luke DiTommaso (COO/lead VFX supervisor), and the name is indicative of the work the company does. Healer explains the significance of the moniker with this scientific analogy, “When elements come together, they form molecules, and their properties actually change. It’s that transformation of properties that results in a sum, and the sum of the parts is greater than the individual components.”
With that in mind, The Molecule started with an office in New York, focusing mainly on graphics and animation. Eventually the facility was awarded television series and feature films, for which the team generates traditional effects integrated into the raw camera footage, resulting in more extensive shots. Today, The Molecule’s mainstay is CGI and compositing — particularly within the television market.
“Our work has increased tremendously. In the early days, when we were doing Rescue Me, for instance, a show might only have 20 visual effects shots, and that was at regular HD 1080 with RGB color space,” says Healer. “Now we’re doing shots in 4K, 6K and 8K, and the show might have 150 VFX shots. And we’re often doing eight shows simultaneously. So yes, the work has scaled a lot over the years.” The same holds true for the computers that are used to do that work. “When we started, computers had only one processor available, and now we’ve got workstations that are 20-core machines out on the floor. That’s enabled a much more expansive workflow and much higher volume of output.”
As the workload at The Molecule expanded, so too did the company itself. Today, The Molecule New York has approximately 50 employees. In 2011, the company expanded to Los Angeles, adding another 16 artists to its roster. The work functions are similar across both locations, with New York handling the CG work (visual effects, experiential design, creatures, set extensions and more), while compositing and on-set supervision is done at both sites.
A good part of Healer’s day is focused on rebuilding the company’s infrastructure to make it stronger. And that involves a lot of code writing, investigating hardware, maintaining connectivity between the two offices and all the workflow implications associated with that. “A particular shot may be sent to our New York office, go to one artist, then be passed on to a different artist at the LA office, and then it might come back to New York for CG,” he explains. “The shots can move around quite a bit.”
Building a Strong Infrastructure
With increased demands, The Molecule had outgrown its earlier hardware infrastructure, which comprised custom-built machines. However, the turnaround for delivery could be three, often four, weeks. “And that just wasn’t working,” Healer says. “A project will come in and the client might add 50 more shots. We’ll then need to hire some artists and will need machines to put them on. So, when we have to buy machines, turnaround is really important.”
As a result, The Molecule began investing in Dell workstations. “We like Dell. It’s a known quantity, and parts replacement is easy,” says Healer. Another overwhelming reason, he maintains, is the Dell Financial Services’ range of finance options, which, for a smaller company like The Molecule, is a big deal.
Today, most of the workstations in use at The Molecule are Dell single-socket machines with eight cores, 16 if there is hyperthreading. However, these 32GB machines are now reaching their limits under the increased workloads. “The motherboards are maxed out,” says Healer. “That’s why we’re in the process of buying new machines.”
At the time of this interview, Healer was finalizing a new deal with Dell that increased the minimum standard workstation configuration. “We’ll have the possibility of dual sockets, 10 cores per socket, and they’re upgradable to 3TB of RAM,” Healer says. “The current plan is to hit 64 gigs with the option to update to 128 or 256 later. RAM is critical because Nuke, Maya and Houdini don’t really use the GPU during rendering.”
In all, The Molecule has purchased nine new Dell Precision 7920 towers and three new Precision 3630 tower workstations with the latest Intel Xeon processors and NVIDIA Quadro graphics cards — a powerful combination in Healer’s opinion. The later expandability of these machines makes them a great option. The Molecule locations run on Linux, making support for that OS critical. Healer calls the NVIDIA Quadro driver support for Linux “fantastic,” while the Dell Precision machines are certified on Linux.
Configuring a Workstation That Works
Workstations at The Molecule are configured with a minimum of two monitors, and some even have three or four. “That configuration works well with an NVIDIA/Dell combination,” says Healer. For now, the facility has a range of monitor brands but is cycling in new Dell 4K monitors. Presently there are eight to 10 UltraSharp 27 4K monitors at the two facilities.
To round out the hardware, the company has two Dell servers with 48 cores each, “which act as a kind of mother brain of our operation,” says Healer. The Molecule also runs Proxmox VE, an open-source platform for enterprise virtualization. “We take those two servers and run Proxmox on them, which allows us to partition off the machine into virtual machines, and then they can act as ‘save’ folders for each other,” he adds. The shop also runs its docker stack inside of Proxmox redundantly on those two servers.
In addition, the studio has a virtual private cloud (VPC) on Amazon Web Services (AWS) that its farm will reach out to when needed. The VPC connection enables The Molecule to start AWS and have it join the studio’s network over the VPN.
Another advantage of the Dell Precision workstations: They are ISV-certified for the most popular independent software applications, including many of the tools being used at The Molecule, including Autodesk’s Maya, SideFX’s Houdini, Foundry’s Nuke, Andersson Technologies’ SynthEyes, Boris FX’s Mocha and Adobe’s After Effects, Illustrator and Premiere. Much of that is linked together using Shotgun project management software. In addition to those tools, The Molecule has scripted various in-house software, including an auto time-tracking system for the artists that make the workflow more efficient.
Addressing Bottlenecks
Despite the tools and steps implemented by The Molecule, bottlenecks can still occur at times. “It’s a combination of things. First, resolution is getting bigger. It used to be that everything was HD, and then we’d occasionally get 4K. Now everything is 4K with the occasional 6K and 8K. Rarely does any HD work — standard 1080 — come in anymore. That causes a problem, because at that point you’re looking at potentially 80MB to 100MB per frame of just raw source material, which can then be painful to move over a network, especially when you’ve got 70 people trying to pull frames at the same time,” he explains.
Healer looks forward to eliminating some of those pain points with the new Dell workstations. “It’s a major obstacle when an artist gets a complex shot and their machine can’t quite keep up. It might be that they spend half the day just waiting for the software,” he says. “We’ve rented a few machines, and based on our experiments, that seems to be a pain point that has been erased [with the new purchase]. So, we’re going to replicate that on this batch of new machines.” In essence, the artists will work on their individual workstations, but then those workstations can also serve as a part of a number of render farms. So if a machine is not in use by an artist, the studio can activate it at night, for instance, when there’s no one there, and put those machines on the farm.
The Molecule at Work
According to Healer, the studio does a lot of location enhancements — adding or deleting trees, snow and water, for example. The Molecule also does a lot of driving comps, cosmetic work and, to a lesser extent, character work — all mostly for episodic series across many of the networks and streaming services, including NBC, CBS, Apple TV+, Hulu, Netflix and HBO.
Currently, The Molecule is working on P-Valley, a new crime drama for Starz that follows strip club dancers working deep in the Mississippi Delta. According to Healer, large parts of the series occur outdoors in a cotton field, and The Molecule was tasked with adding CG cotton to the field for hundreds of shots. The artists recreated the cotton in CG, randomized it and then populated the fields with several variations of the cotton. Compositing the cotton into the field was a difficult task — close to 60 to 70 percent of the final frame had been replaced by the studio’s work. “And if we do our job right, no one will even know there’s CG in the shots,” says Healer.
The studio is also working on a new show for Apple TV+ called Dickinson, a period piece centered on the poet Emily Dickinson, which is now available. Among the work for that series, The Molecule crafted dream sequences with horses made out of smoke, which required volumetric, particle-based simulations. And because the frames had to be processed sequentially, the sim could only run on one machine. “So in that circumstance, you really need a super-powerful single workstation,” Healer says.
While the artists at The Molecule likely see the “art” side of the work they do, it’s people like Healer who must also consider the science aspect of the work. And without it, the artists wouldn’t be able to work as efficiently, if at all. It’s just elemental logic.
|