By Alyssa Heater

Little-known fact: the iconic Muppets rock band Dr. Teeth and the Electric Mayhem, who debuted on the pilot for The Muppet Show in 1975, never actually recorded an album… until now. The Muppets Mayhem, now streaming on Disney+, follows the band as it attempts to record its first album and the adventures that ensue while navigating the modern music industry and collaborating with some of today’s top artists.

Craig Kief and friends

We sat down with cinematographer Craig Kief, ASC, and virtual production supervisor Brian Doman of Monolith Studios to discuss the benefits and challenges of shooting in an LED volume for the latest Muppet adventure.

Craig, you’ve shot for feature films, TV series, shorts, music videos and several Muppets projects. How does shooting with puppets (and their puppeteers) as well as human actors differ from other productions you’ve worked on?
Craig Kief: It’s very different. Photographically at least, I try not to treat the puppets as anything special and just make everything as real, natural and homogenized as possible. I never want it to feel like a puppet show. These are real people as far as I’m concerned, and I want to portray them in a realistic manner. That ethos goes all the way back to the very beginning with Jim Henson. They always try to do as much in-camera naturally as possible and not lean on visual effects. Making it as natural and seamless as possible has always been my perspective, but underneath the frame, there’s a whole lot going on.

One of the big differences when we’re on a set onstage is that the sets are raised 4 feet off the ground so that the puppeteers can stand at zero on the stage floor. When they raise their right hands with the puppets on, that puts them at a natural height within the set. The human actors stand at the 4-foot level, which then puts them at their natural height within the set. The cameras will live somewhere in between. Blocking-wise, particularly once actors start moving around within a scene, it becomes a challenge figuring out where and where not to build steel deck and how to make the camera move around in a very complicated, three-dimensional puzzle of how everything needs to be laid out.

It is also a technical challenge because everything shifts. For example, lighting: Normally on a set, lighting starts at approximately 8 feet off the ground. Well, now it starts at 12 feet off the ground. Everything must be rigged differently to do this, and it takes more time to do everything. It’s very different than working on a normal set.

On-location, we’ll have the puppeteers working off the normal ground. We have all sorts of ways of moving the puppeteers: They’ll roll around on modified dollies like we use for hand-held camera operating. We’ve built large sled platforms and pulled them across the desert floor. We’ve rigged puppeteers to wheelchairs – all sorts of things.

Shooting in cars is another challenge. We basically cram the puppeteers down into the floorboard area of the car. We’ve destroyed cars before by hacking them to fit people into tight corners. We’ve even had rear drive units in vehicles, so it appears that the puppets are actually driving the cars. One time we made a remote-controlled golf cart for Kermit to drive. We’ve used every single trick in the book to pull off the illusion.

During prepro, what made you decide that virtual production would be a good solution for The Muppets Mayhem?
Kief: We went into Mayhem having come off the previous Muppets project Muppets Haunted Mansion, which was decided to be a virtual production before I was brought on board. The basic reason was that the idea of Muppets Haunted Mansion had been floating around for over a year, and really the biggest barrier to it being made was the cost of building a real haunted mansion. Once they found a way to do it virtually, they saved a tremendous amount of money in set construction by instead shooting it in an LED volume, and that was a major reason why Disney+ was able to give us the green light for the project.

The movie turned out great, and the virtual production part of it looks fantastic. Going into Mayhem, which we shot around a year and a half later, everybody was very pro-volume — in particular, series co-creator and lead puppeteer Bill Barretta. When we were going through the scripts, his vision was to do tons of stuff in a volume. The reality is that there are plenty of great reasons to shoot in a volume and plenty of great reasons not to shoot in a volume. For example, it can be difficult to fit large-volume work onto a traditional episodic television schedule and budget. Ultimately, we only shot one episode (Track 5: “Break on Through”) in a volume because many of our sets needed to be real. We went back to them multiple times throughout the season, and it made more sense to build some of the sets that were originally pitched to us as virtual production.

The scenes that we shot in virtual production take place out in Joshua Tree. First of all, nobody wants to go out and work at night in the desert for a week. Additionally, moving the company out there would have been enormously expensive on its own, so those are compelling reasons to do this work in a volume. As we were exploring the possibility of this, someone asked, “Craig, what would you prefer?” and I realized that in a real desert, I would be able to afford to light a few square acres, whereas on a volume, I could light the entire desert. I could see the stars. I could see the mountains with a little light on the horizon. It created such a wonderful creative opportunity with unlimited possibilities.

In addition to the difference between lighting in the desert or lighting in a volume, there also must be differences between lighting with puppets in addition to human actors. I’m sure you have to take into consideration skin tone, etc. What is it like lighting when you have actors of the puppet genre as well as of the human genre?
Kief: That’s a question I get asked a lot. I try not to treat the puppets any differently than the human actors. And my lighting does the same; it doesn’t treat anybody special. I will say that there are a few very dark puppet characters with deep-colored fur or deep-set eyes for which I’ll add a little fill light just for them. Other than that, I really don’t treat them as anything special.

That said, my longtime colorist, Chris Boyer at Picture Shop, and I do have color-grading techniques where we’ll lift people or puppets up with windows or take people down to balance out some of the reflectivity because the materials that the puppets are made with soak up a great deal of light. I’ll do minor adjustments, but really it’s working with a scalpel instead of a hammer.

Craig Kief

One big difference that I do make with the puppets happens in final color. The puppets have very bright, saturated colors, and they are all over the spectrum, especially in HDR. We spend a great deal of time treating the colors so that they come across more naturally and more homogenized with the colors and the costumes of the human actors. We also take great care to ensure that the colors stay true regardless of the lighting or environment. Our highest priority in final color is to make sure that these iconic characters look the same as everybody knows them to look because everybody knows exactly what they look like.

What camera and lenses did you use for the project?
Kief: I shot this on Red’s new V-Raptor camera with Panavision’s prototype VA Series prime lenses. They’re gorgeous, large-format glass because the V-Raptors are a large-format sensor, which is one of the things that helped us on the volume. You have to take great care to keep the wall slightly out of focus so that it doesn’t moiré, and the large format helps to get us there faster. I specifically selected the Red after testing a few different cameras with the actual puppets in a lighting setup on a stage at Panavision.

We ran all the cameras that I tested through final color, I looked at them, then we chose the Red because of the color rendition. I really liked what it did to the colors of the characters. It gave us a great starting point to go into final grade and do the adjustments we wanted.

Brian Doman

Tell me about the volume that was created for this episode. Any dimensions or power requirements? Anything that made it specifically unique to this production?
Brian Doman: With input from Craig, Monolith designed and built a 58-foot-wide semicircular volume that featured a compound curve of 2.25 to 3.0 degrees — a relatively shallow bend to suit the production’s requirements for the Joshua Tree environment. We deployed ROE Black Pearl 2 V2 panels with Brompton processing for a total of six outputs — basically six UHD view ports to drive the whole wall. Stype RedSpy camera tracking was the choice for this project.  It’s an inside out system that utilizes special stickers that are applied to the CB5 panels in the ceiling. We had two units onstage so that Craig could use them selectively on cameras A, B and C.  The CB5 LED ceiling panels provided lighting from above. That product can deliver 6,000 nits of brightness, and they worked great.

Kief: The hardware that Monolith uses is the best. It’s what ILM uses. It’s what Amazon uses. All of the premium volumes are using this same hardware because what we’re asking these extremely complex machines to do is so processor-intensive that we’re barely pulling it off with the very best. Several disparate systems have to work together and at an extremely high level of performance, and if they don’t, you start to have more problems in-camera. It is important when you are evaluating a volume to research what the machine is made of before you decide whether it will be well-suited for your production needs.

As far as on-set lighting on this particular load, I didn’t use the volume for very much of the foreground lighting. We had one scene that was shot daytime at dusk, so for that, I did use the environmental light of the volume quite a bit. I also used a couple of light cards and some negative on the volume walls and ceiling. For the desert at night, I used a pretty large lighting setup. Because it was a very dark environment, I couldn’t use the wall to light the foreground because it would have spilled too much light onto the other panels, which would have milked the black levels. The studio lighting that I used was very controlled so it would only affect the real set elements in the foreground and would avoid leaking onto the wall much.

You spoke about how important it is to make sure the colors of these Muppets are what audiences expect them to be. As far as virtual production and the post process, do you feel this helps streamline final color, or is there a bit of a learning curve with the colorist since virtual production is not necessarily the “traditional” way of shooting?
Kief: One of the biggest challenges to prepare for when going into a volume shoot is understanding what the panels are capable of doing and what they are not capable of doing because it will affect final color. Virtual production was not the intended use for the panels we are working with. The technology was developed for concerts, sports arenas and billboards, and they are meant to look good to the eye as well as look good on a camera.

To accomplish this, the panel manufacturers have made the panels out of a very narrow band of LED emitters — a red, a green and a blue emitter — to create the best visual display medium. When used as an emissive light source in a volume, the quality of light that is emitted onto your foreground elements is lacking in the spectrum. The panels are almost completely lacking, for example, in the orange/yellow part of the spectrum, the cyan part of the spectrum and the very low-end blues and high-end reds. Anything that contains those colors in the foreground will be deficient and won’t reproduce very accurately on-camera… unlike it would with our full-spectrum LED studio lighting fixtures.

This is especially true for skin tones. Skin is very complicated, and it requires a full spectrum of light to render properly on-camera. When you deny it a full spectrum, you get color shifts, and things just don’t look right on skin. The most noticeable effect when shooting on a volume is that skin takes on a reddish coloration, and people can look ruddy. One misconception is that a volume does all the lighting for you. That’s absolutely not the case and should not be the case because of these issues. You certainly want the volume to do the heavy lifting of the lighting and be in the general area of the tonality because that is one of the huge advantages of using a volume instead of a greenscreen — the lighting is the same between the foreground and the environment. But you still need to supplement with studio lighting to resolve some of these issues.

All of these factors affect final color. If you walk onto a volume shoot unprepared for this, then you are going to have trouble in final color trying to make things look right. So be prepared going into the shoot by thinking about your final color in advance. The only thing that matters when you’re shooting on a volume is what you see in the camera; it doesn’t matter at all what it looks like to your eye. Anybody can walk onto the volume and examine with their eyes the color, luminance values and the light that is falling onto the set. A great deal of testing needs to be done to ensure that the environment, the foreground, the set pieces, the costumes are all looking good through the camera because it can be quite different than what you see with your eye. This all needs to be considered before going into the volume because you do not want to discover these issues on a very expensive stage day. It is also important to be able to make finite color adjustments to the wall itself, as well as to the camera and the lighting, so everything looks cohesive through the camera. The magic trick becomes really cool when you get to that final moment when everything starts blending together and just looks right. These color adjustments are important to do on the shoot day because they impact your success in the final grade, where, hopefully, you are just adding that last 10% of polish on top of the capture, as you normally would with a real set or a real location.

Many of the technical complications that we come across when working on a volume are a result of this technology being in its infancy. Another frequent issue to consider is moiré, which is a problem when you’re working with 2.8mm and higher pixel-pitch panels. When you get down into 1.9, 1.5 pixel-pitch panels, moiré is not really an issue, and it is almost impossible to make the camera moiré. Panels that have those pixel pitches already exist and are going to be more and more common in panels designed specifically for virtual production. Because the technology is evolving extremely rapidly, soon these will no longer be factors that even need to be considered.

Were there any sequences within The Muppets Mayhem that were particularly rewarding to shoot?

Kief: I would say everything we did on the volume; these scenes are very special and very funny. We do things in this episode that have never been done with the Muppets before. Everything in this experience was really a blast to shoot, and it turned out so great. I’m really excited for people to see it.

Brian, Craig, is there anything else you’d like to share?
Doman: Monolith Studios entertains interest from many different productions, and while we have worked with episodic productions numerous times, this was my favorite experience of them all. It is so fun to be on-set with the Muppets, and Craig was amazing to work with. He knows technically what he wants, he has taken the time to understand the process, and he knows what can be achieved. He’s formed a rapport very quickly with the artists on the Monolith team who were responsible for executing the creative changes he needed for the wall itself, and all that went into making it such an enjoyable experience for me and my team.

Alyssa Heater is a writer and marketer in the entertainment industry. When not writing, you can find her front row at heavy metal shows or remodeling her cabin in the San Gabriel Mountains.