By Randi Altman
The writing and directing duo of Joel and Ethan Coen have taken on the American Western with their new Netflix film, The Ballad of Buster Scruggs. This offering features six different vignettes that follow outlaws and settlers on the American frontier.
It stars the Coen brothers’ favorite Tim Blake Nelson as Buster, along with Liam Neeson, James Franco, Brenden Gleeson and many other familiar faces, even Tom Waits! It’s got dark humor and a ton of Coen quirkiness.
Alex Lemke (middle) on set with the Coen brothers.
For their visual effects needs, the filmmakers turned to New York-based East Side Effects co-founders and VFX supervisors Alexander Lemke and Michael Huber to help make things look authentic.
We reached out to visual effects supervisors Lemke and Huber to find out more about their process on the film and how they worked with these acclaimed filmmakers. East Side Effects created two-thirds of the visual effects in-house, while other houses, such as The Mill and Method, provided shots as well.
How many VFX shots were there in total?
Alexander Lemke: In the end, 704 shots had digital effects in them. This has to be a new record for the Coens. Joel at one point jokingly called it their “Marvel movie.”
How early did you get involved? Can you talk about that process?
Michael Huber: Alex and myself were first approached in January 2017 and had our first meetings shortly thereafter. We went through the script with the Coens and designed what we call a “VFX bible,” which outlined how we thought certain effects could be achieved. We then started collecting references from other films or real-life footage.
Did you do previs?
Lemke: The Coens have been doing movies for so long in their own way that previs never really became an issue. For the Indian battles, we tried to interest them in the Ncam virtual camera system in combination with pre-generated assets, but that is not their way of doing a film.
The whole project was storyboarded by J. Todd Anderson, who has been their go-to storyboard guy since Raising Arizona. These storyboards gave a pretty good indication of what to expect, but there were still a lot of changes due to the nature of the project, such as weather and shooting with animals.
What were some of the challenges of the process and can you talk about creating the digital characters that were needed?
Huber: Every story had its own challenge, ranging from straightforward paintouts and continuity fixes to CG animals and complex head replacements using motion control technology. In order to keep the work as close to the directors as possible, we assembled a group of artists to serve as an extended in-house team, creating the majority of shots while also acting as a hub for external vendor work.
In addition, a color workflow using ACES and FilmLight Baselight was established to match VFX shots seamlessly to the dailies look established by cinematographer Bruno Delbonnel and senior colorist Peter Doyle. All VFX pulls were handled in-house.
Lemke: The Coens like to keep things in-camera as much as possible, so animals like the owl in “All Gold Canyon” or the dog in “Gal” were real. Very early on it was clear that some horse falls wouldn’t be possible as a practical stunt, so Joel and Ethan had a reel compiled with various digital horse stunts — including the “Battle of the Bastards” from Game of Thrones, which was done by Iloura (now Method). We liked that so much that we decided to just go for it and reach out to these guys, and we were thrilled when we got them on board for this. They did the “dog-hole!” horse falls in the “The Gal Who Got Rattled” segment, as well as the carriage horses in “Mortal Remains.”
Huber: For the deer in “All Gold Canyon,” the long-time plan was to shoot a real deer against bluescreen, but it became clear that we might not get the very specific actions Joel and Ethan wanted to see. They were constantly referring to the opening of Shane, which has this great shot of the titular character appearing through the antlers of a deer. So, it became more and more clear it would have to be a digital solution, and we were very happy to get The Mill in New York to work on that for us. Eventually, they would also handle all the other critters in the opening sequence.
Can you talk about Meal Ticket’s “artist” character, who is missing limbs?
Lemke: The “Wingless Thrush” — as he is referred to on a poster in the film — was a combined effort of the art department, special effects, costume design, VFX and, of course, actor Harry Melling’s incredible stamina. He was performing this poetry while standing in a hole in the ground with his hands behind his back, and went for it take after take, sometimes in the freezing cold.
Huber: It was clear that 98% of all shots would be painting out his arms and legs, so SFX supervisor Steve Cremin had to devise a way to cut holes into the set and his chair to make it appear he was resting on his stumps. Our costume designer, Mary Zophres, had the great idea of having him wear a regular shirt where the longs sleeves were just folded up, which helped with hiding his arms. He wasn’t wearing any blue garment, just black, which helped with getting any unnecessary color spill in the set.
Alex was on set to make sure we would shoot clean plates after each setup. Luckily, the Coen brothers’ approach to these shots was really focusing on Harry’s performance in long locked-off takes, so we didn’t have to deal with a lot of camera motion. We also helped Harry’s look by warping his shoulders closer to his body in some shots.
Was there a particular scene with this character that was most challenging or that you are most proud of?
Lemke: While most of the paintout shots were pretty straightforward — we just had to deal with the sheer amount of shots and edit changes — the most challenging parts are when Liam Neeson carries Harry in a backpack up the stairs in a brothel. He then puts him on the ground and eventually turns him away from the “action” that is about to happen.
We talked about different approaches early on. At some point, a rig was considered to help with him being carried up the stairs, but this would have meant an enormous amount of paint work, not to mention the setup time on a very tight shooting schedule. A CG head might have worked for the stairs, but for the long close up shots of Harry — both over a minute long, and only with very subtle facial expressions — it would have been cost prohibitive and maybe not successful in the end. So a head replacement seemed like the best solution, which comes with its own set of problems. In our case, shooting a head element of Harry that would match exactly what the dummy on Liam’s back and on the ground was doing in the production plates.
We came up with a very elaborate set up, where we would track the backpack and a dummy in the live-action photography in 3D Equalizer. We then reengineered this data into Kuper move files that would drive a motion control motion base combo.
Basically, Harry would sit on a computerized motion base that would do the turning motion so he could react to being pushed around. This happened while the motion control camera would take care of all the translations. This also meant our DP Bruno had to create animated lighting for the staircase shot to make the head element really sit in the plate.
We worked with Pacific Motion for the motion control. Mike Leben was our operator. For the NAC effects for the motion base, Nic Nicholson took care of this. Special thanks goes out to Christoph Gaudl for his camera and object tracking, Stefan Galleithner for taking on the task of converting all that data into something the camera and base would understand, and Kelly Chang and Mike Viscione for on-set Maya support.
Of course, you only get an element that works 80% of the way — the rest was laborious compositing work. Since we put the motion base to its speed limits on the staircase shot, we actually had to shoot it half speed and then speed it up in post. This meant a lot of warping/tracking was needed to make sure there was no slippage.
Michael Huber
The dummy we used for the live-action photography didn’t have any breathing movement in it, so we used parts of Harry’s bluescreen plates as a guideline of how his chest should move. These tricky tasks were expertly performed mainly by Danica Parry, Euna Kho and Sabrina Tenore.
Can you talk about how valuable it is being on set?
Huber: It is just valuable to be on set when the call sheet calls for a greenscreen, while we really need a bluescreen! But joking aside, Joel and Ethan were very happy to have someone there all the time during the main shoot in case something came up, which happened a lot because we were shooting outdoors so much and we were dependent on the weather.
For the opening shot of Buster riding through Monument Valley, they were thinking of a very specific view — something they had seen on a picture on the Internet. Through Google Maps and research, Alex was able to find out the exact location that picture was taken. So, on a weekend when we weren’t shooting, he packed up his family and drove up to the Valley to shoot photographs that would serve as the basis for the matte painting for the first shot of the film — instead of going there with a whole crew.
Another instance being on set helped would be the scene with Tom Waits in the tree — the backgrounds for these bluescreen shots were a mixture of B camera and Alex’s location photography while in Colorado. Same goes for the owl tree backgrounds.
What tools did East Side use on the film?
Huber: For software we called on Foundry Nuke (X & Studio), Boris FX Mocha Pro and Side Effects Houdini. For hardware we used HP and SuperMicro workstations running Linux. There was also proprietary software such as using Houdini digital assets for blood simulations.
We were using Autodesk Shotgun with a proprietary connection to Nuke that handled all our artist interaction and versioning, including automatically applying the correct Baselight grade when creating a version. This also allowed us to use the RV-Shotgun integration for reviewing.
Can you talk about the turnaround times and deadlines?
Lemke: Working on a Coen brothers film means you don’t have a lot of things you normally have to deal with — studio screenings, trailers, and such. At the same time, they insisted on working through the stories chronologically, so that meant that the later segments would come in late in the schedule. But, it is always a great experience working with filmmakers who have a clear vision and know what they are doing.
