By Karen Moltenbrey

A couple of decades or so ago, studios needed the exceptional power of a machine offered by the likes of SGI for complex visual effects. Non-specialized PCs simply were not cut out for this type of work. But then a sea change occurred, and suddenly those big blue and purple boxes were being replaced with options in the form of workstations from companies such as Sun, DEC, HP, IBM and others, which offered users tremendous value for something that could conveniently fit on a desktop.

Those hardware companies began to duke it out, leading to the demise of some and the rise of others. But the big winners in this early war for 3D content creators’ business were the users. With a price point that was affordable, these workstations were embraced by facilities big and small, leading to an explosion of 3D content.

Here, we look at two VFX facilities that have taken different approaches when it comes to selecting workstations for their digital artists. NYC’s Bonfire, a boutique studio, uses a range of Boxx and custom-built machines, along with iMacs. Meanwhile, Digital Domain, an Oscar-winning VFX powerhouse, recently set up a new site in Montreal outfitted with Dell workstations.

Dave Dimeola

Bonfire
Bonfire is a relative newcomer to the industry, founded three years ago by Flame artist Brendan O’Neil, who teamed up with Dave Dimeola to get the venture off the ground. Their goal was to create a boutique-style base for those working in post production. The front end would comprise a beautiful, comfortable space where Autodesk Flame and motion graphics artists could work and interact with clients in comfortable suites within a townhouse setting, while the back end would consist of a cloud-based pipeline.

“We figured that if we combined these two things — the traditional boutique shop with the client experience and the power of a cloud-based pipeline — we’d have something,” says Dimeola, whose prior experience in leveraging the cloud proved invaluable in this regard.

Soon after, Peter Corbett, who had sold his Click 3X creative digital studio in 2018, agreed to be part of Bonfire’s advisory board. Believing Dimeola and O’Neil were on to something, Corbett came aboard as a partner and brought “some essential talent” into the company as well. Currently, Bonfire has 11 people on staff, with great talent across the gamut of production and post — from CG and creative directing to producing and more. One of the first key people who Corbett brought in was managing director Jason Mayo.

And thanks to the company’s unconventional production pipeline, it is able to expand its services with remote teams as needed. (Currently, Bonfire has more than 3,000 vetted artists in its network, with a core group of around 150 who are constantly on rotation for work.)

“It’s a game-changer in the current climate,” Dimeola says of the company’s setup. The group is performing traditional post work for advertising agencies and direct to client. “We’re doing content, commercials, social content and brand films,” says Dimeola, “anything that requires storytelling and visual communication and is design-centric.” One of the studio’s key offerings is now color grading, handled by colorist Dario Bigi. In terms of visual effects, Dimeola notes that Bonfire can indeed make animals talk or blow things up, although the VFX work Bonfire does “is more seamless, artful, abstract and weird. We get into all those facets of creation.”

Bonfire has approximately 10 workstations at its location in New York and is expanding into the second floor. The company just ordered a new set of customized PCs with Nvidia GeForce RTX 2070 Super graphic cards and some new ultra-powerful Apple iMacs, which will be used for motion graphics work and editing. The core software running on the machines includes the major industry packages: Autodesk’s Maya, Maxon’s Cinema 4D, Side Effects’ Houdini, Autodesk’s Flame, Foundry’s Nuke and Adobe’s Creative Suite, in addition to Thinkbox’s Krakatoa for particle rendering and Autodesk’s 3ds Max for architectural work. Cinema 4D motion graphics software and the Adobe software will run on the new iMac, while the more render-intensive projects within Maya, Houdini and Nuke will run on the PC network.

As Dimeola points out, workstations have taken some interesting turns in the past two to three years, and Bonfire has embraced the move from CPU-based systems to ones that are GPU-based. As such, the company’s PC workstations — a mix of Boxx and custom-built machines (with an AMD Threadripper 2950X CPU and a pair of Asus GeForce RTX 2080 Ti video cards, along with significant memory) — contain powerful Nvidia Quadro RTX 1080 cards. He attributes Bonfire’s move in processing to the changing needs of CGI rendering and lighting, which are increasingly relying on GPU power.

“We still use CPU power, however, because we feel some of the best lighting is still being done in the CPU with software like [Autodesk’s] Arnold,” Dimeola contends. “But we’re flexible enough to be using GPU-based lighting, like Otoy’s OctaneRender and Maxon’s Redshift for jobs that need to look great but also move very quickly through the pipeline. Some shops own one way of rendering, but we really keep a flexible pipeline so we can pivot and render just about any way we want based on the creative, the look, the schedule. It has to be very flexible in order for us to be efficient.”

The media will work off the SAN, a communal server that is partitioned into segments: one for the CGI, another for the editing (Flame) and a third for color (Blackmagic DaVinci Resolve). “We partitioned a cloud section for the server, which allows us to have complete control on how we sync media with an external team,” explains Dimeola. “That’s a big part of how we share, collaborate and move assets quickly with a team inside and outside and how we scale for projects. This is unique for Bonfire. It is the innovative part of the post production that I don’t think any other shops are really talking about at this time.”

In addition to the local machines and software, Bonfire also runs applications on virtual machines in the cloud. The key, says Dimeola, is knowing how to create harmony between the internal and external infrastructures. The backbone is built on Amazon Web Services (AWS) and Google Cloud Platform (GCP) and functions much the same way as its internal pipeline does. A proprietary project tracker built by Bonfire enables the teams to manage shots and assets; it also has an array of services and tools that help the staff efficiently manage projects that vary in complexity and scale.

Brooklyn Nets

“It’s no single piece to our pipeline that’s so innovative; rather, it’s the way that we’ve configured it between our talent and infrastructure,” says Dimeola, noting that in addition to being able to take on big projects, the company is able to get its clients what they need in real time and make complete changes literally overnight. Dimeola recalls a recent project for Google requiring intensive CGI fluid simulations. The team sat with the client one day to work out the direction and was able to post fresh edits, which included rendered shots, for the client the very next morning. “[In a traditional setup], that never would have been possible,” he points out.

However, getting the best deal on the cloud services requires additional work. “We play that market like the stock market, where we’re finding the best deals and configurations based on our needs at the time,” Dimeola says, and the result is an exponential increase in workflow. “You can ramp up a team and be rendering and working 24/7 because you’re using people in different time zones, and you’re never down a machine for rendering and working.”

Best of all, the setup goes unnoticed by the customer. “The client doesn’t feel or see anything different,” says Dimeola. That is, with one exception: “a dramatic change in the cost of doing the work, particularly if they are requiring a lot of speed.”

Digital Domain Montreal
A longtime creative and technological force in the visual effects industry, Digital Domain has crafted a range of work spanning feature films, video games, commercials and virtual reality experiences. With global headquarters in LA, plus locations in Vancouver, Beijing, Shanghai and elsewhere around the globe, the studio has been the driving force behind many memorable and cutting-edge projects, including the Oscar-winning The Curious Case of Benjamin Button and more. In fact, Digital Domain is known for its technological prowess within visual effects, particularly in the area of realistic digital humans, recently recreating a photoreal 3D version of Martin Luther King Jr. for a groundbreaking immersive project.

Michael Quan

A year ago, Digital Domain expanded its North American footprint by opening an office in Montreal, which celebrated its grand opening this past February. The office has approximately 100 employees, with plans to expand in the future. Most of the work produced by Digital Domain is shared by its five worldwide studios, and that trend will continue with Digital Domain Montreal, particularly with the LA and Vancouver offices; it also will tackle regional projects, focusing mostly on features and episodic content.

Setting up the Montreal site’s infrastructure fell to Digital Domain’s internal IT department, including senior systems engineer Michael Quan, who helped outfit the facility with the same classes of machines that the Los Angeles and Vancouver locations use: the Dell Precision R7920 and R7910 rack workstation PCs. “All the studios share common configuration specifications,” he notes. “Having a common specification makes it tremendously easy to move resources around when necessary.”

In fact, the majority of the machines were purchased approximately during the third quarter of 2019. Prior to that, the location was started up with resources from the facility’s other sites, and since they are using a common configuration, doing so did not present an issue.

Quan notes that the studio is able to cover all aspects of typical VFX production, such as modeling, rigging, animation, lighting, rotoscoping, texture painting, compositing and so forth, using the machines. And with some additional hardware, the office can also leverage those workstations for dailies review, he adds. As for the software, Digital Domain runs the typical programs: Autodesk’s Maya, Foundry’s Mari and Nuke, Chaos’ V-Ray, Maxon’s Redshift, Adobe’s Photoshop and so on, in addition to proprietary software.

Terminator: Dark Fate

As Quan points out, Digital Domain has specific requirements for its workstations, aside from the general CPU, RAM and hard drive specs. The machines must be able to handle the GPUs required by Digital Domain along with additional support devices. While that might seem obvious, when a requirement comes into play, it reduces the number of systems that are available for evaluation, he points out. Furthermore, the workstations must be rack-mountable and of a “reasonable” size (2U) to fit within the data center as opposed to deskside. Also, since the workstations are deployed in the data center, they must be manageable remotely.

“Preferably, it is commodity hardware, meaning using a vendor that is stable, has a relatively large market share and isn’t using some exotic technology,” Quan says, “so if necessary, we could source from secondary markets.” Unfortunately, the company learned this the hard way in the past by using a vendor that implemented custom power and management hardware; the vendor exited the market, leaving the studio without an option for repair and no secondary market to source defective parts.

Just how long Digital Domain retains its workstations depends on their performance effectiveness: If an artist can no longer work due to a resource inefficiency, Quan says, then a new round of hardware specification is initialized.

“The workstations we use are multi-processor-based, have a relatively high amount of memory and are capable of running the higher-performing professional GPUs that our work requires,” he says. “These days, ‘workstations’ could mean what would normally be called gaming rigs but with more memory, a top-end GPU and a high-clock-speed single processor. It just depends on what software will be used and the hardware configuration that is optimized for that.”

Lost in Space, Season 2

As Quan points out, graphics workstations have evolved to where they have the same capabilities as some low- to mid-class servers. “For example, the Dell R7910/R7920 that we are using definitely could be used as servers, since they share the same hardware capability as their server class,” he says. “It used to be that if you wanted performance, you might have to sacrifice manageability and footprint. Now there are systems deployed with one, eight and 10 GPU configurations in a relatively small footprint, which is a fully remotely manageable system in one of our data centers.” He predicts that workstations are evolving to a point where they will just be a specification. “In the near future, it will just be an abstract for us. Gone will be the days of one workstation equating to one physical system.”

According to Quan, the Montreal studio is still ramping up and has several major projects on the horizon, including feature films from Marvel, Sony, 20th Century Studios and more. Some of Digital Domain’s more recent work includes Avengers: Endgame, Lost in Space (Season 2), Terminator: Dark Fate and several others. Globally, its New Media and Digital Humans groups are doing incredible things, he notes, and the Ads/Games Group is producing some exceptional work as well.

“The workstations at Digital Domain have constantly evolved. We went from generic white boxes to Tier 1 systems, back to white boxes, and now to a more sophisticated Tier 1 data center-targeted ecosystem. With the evolutionary steps we are taking, we are iterating to a more efficient management of these resources,” Quan says. “One of the great advantages of having the workstations placed in a remote location is the security aspects. And on a more human level, the reduction of the fan noises and the beeps all those workstations would have created in the artist locations is notable.”

Karen Moltenbrey is a veteran writer, covering visual effects and post production.