By Samantha Uber

For a Flame editor or any sort finishing editor working in 16-bit EXR or DPX large-format, long-form image sequences, it is important that we are on high-speed shared storage — usually a SAN that has been rated for multiple streams of 4K by our engineers. This is incredibly important so I can be working in the editorial system and playing down as the colorist is grading with the clients on the same data. This could be happening at the same time as our deliverables folks are also accessing those same files to create confidence checks or various other sub-vendor files, and all need to have good performance for our various processes.

Also, I am frequently updating live into the color session, which simultaneously updates that data to all systems pointed at it, which of course is only possible on shared storage.

While I do not specifically choose or design the shared storage we work on, I do judge the requirements of the storage per project based on the size of the source files, the amount of footage, the finishing deliverables and the work style of the client. Interestingly, I think this is often overlooked — two clients can shoot the exact same 8K Red, but the storage requirements of each project can be extremely different.

Film
For example, if I have a scripted, graded dailies film shot on 8K Red, with maybe a small amount of other similar camera formats mixed in, I can expect a few things. Because graded dailies have occurred, likely the footage has been professionally cataloged and archived to LTO tape, and I know the Avid bin metadata is solid and correctly refers back to the source masters on that tape. This would mean only the exact selects for the two-hour feature would be brought online when needed, and rapid restore of other elements in the event of a reconform would be possible by the data team. This means overall source footage would not need to be fully restored in totality to our SAN, so access to the source footage is rapid and therefore original masters can be deleted and restored quickly.

For this environment, I would initially estimate about 20TB for overall feature finishing, depending on the number of VFX and versions of the film, and see how it goes from there. As a side note, if there are HDR finishing requirements as well as P3, XYZ TIFF files and/or Rec. 709 masters, then the sources/graded renders must be at least 16-bit, and another whole copy of the film must exist per color space — pushing the storage requirements higher as each master format is added.

Episodic
Using that same source footage scenario — 8K Red for a 10-episode scripted, graded dailies TV show for a premium network — very different storage requirements could emerge, and not just because of the overall length of the show. Today’s premium shows are generally finished as an overall film, not as individual episodes completely separately, so it’s possible you could be on the third reconform of 101 and the fifth reconform of 103 while initially conforming 106, with four more episodes to go before 101 is ever delivered. This means all 10 episodes will be online with their grades at some point, which is different than a “deliver an episode, delete another episode, start another” model of (mostly) the past.

Sam with her dog Ginny

In this scenario, constant reconforms and initial conforms are happening for every episode, along with likely frequent large VFX pulls. To constantly restore and delete media in this type of rapid turnaround schedule is extremely taxing on the data department and can lead to delays in editorial and color. So, if possible, I would elect to put this type of project on a two-SAN system: a regular, fast 4K production SAN with the restored source on a slower, larger and less performance-intensive nearline SAN. The nearline SAN, or NAS, is a less expensive and overall larger parking lot that essentially provides separate, instantaneous access to the source media on a volume other than the production SAN.

Using this two-SAN model keeps the amount of media as low as possible on the production SAN and gives the most rapid turnaround for these types of clients without taxing the facility as much. Of course, this is still a very large amount of storage and a true luxury.

I have had clients provide their own NAS for me to connect to the facility so that we can implement this type of workflow, and I have also had the facility buy or rent storage for this option for the duration of a series. In all, for a 10-episode 4K job — delivering 10 episodes at once with likely large reconforms and many VFX pulls — I would start the estimate at 100TB of production SAN storage for overall job finishing. Depending on the amount of source footage, the nearline volume could require 100TB to 300TB of “parking lot” storage for the source masters.

I really do enjoy working this way. As mentioned earlier, I prefer to think of a series as one large film, not 10 unique episodes, as I think the creativity involved in working this way brings out the best in the project. I am frequently moving scenes from one episode to another to give the strongest possible storyline. All that creativity takes a lot of storage space to work!

Docs
Finally, a general third type of job, the documentary, can really be a mixture of the above two scenarios or a unique third. Documentaries, especially indie docs, often have been shooting and self-managing their data for years. This means the client might have many hard drives of source material of various ages while also having some graded dailies of their “talking head” footage and tons and tons of archival.

The unique qualities of these jobs are often that irregular file names exist and no LTOs/data cataloging has occurred, as graded dailies weren’t necessary on most of the footage. Usually this means a variety of assistant editors have imported the source footage directly into the Avid (hopefully, Avid!) for many years. As a result, sometimes an automated EDL restore process is not possible, so I really lean into the robustness of the Avid bin database to reorganize metadata and find and restore this footage.

Depending on the documentary, its length, age of subject matter and amount of time being edited, storage requirements can vary widely. Sometimes it is possible to pull the footage right off of the client drives, keeping them mounted up for the duration of the job. And sometimes it is desired/more efficient to copy all sources provided by the client to a nearline SAN for sorting and reorganization, even before the online begins. However, nearlines are a luxury, and documentaries can use a tremendous amount of data, especially if frame-rate conversions and archival up-conversions are also occurring. For these type of clients, the best way to determine the storage requirements of the job is to do an individual investigation into the Avid project bin, catalog the source drives and look at their organization, and consult with the client on their editorial and scheduling needs.

While some mixture of the above three exists, and sometimes you use an additional “graded master-only SAN,” these are the general basics for how I think about storage for individual projects and how I guide jobs.

You don’t need all that storage initially, but over time, as the project grows and nears completion, of course you might need more storage. At completion, usually reconform versions are pared down, and the project is as lean as it can be for delivery and archive.

As a finishing editor, I really do feel about 50% of my job is thinking about storage and organization, keeping it as lean as possible for minimal space usage, and having a clear vision of the current master of the project.

Samantha Uber is a senior finishing editor at Picture Shop Post, working on Autodesk Flame. She previously held positions at Deluxe and Technicolor She is a graduate of NYU Tisch and loves her Jersey Shore Victorian home, her four Afghan Hound dogs and riding around in her 1974 VW bus.