The Soul of Nonlinear Editing

I keep thinking about Tom Ohanian’s series on the State of Digital Nonlinear Editing. Specifically these paragraphs in Part 10:

Content that is recorded will then be processed by a variety of AI application suites. Each suite will provide different functionality (e.g. tonal analysis, speech-to-text, etc.) based on the characteristics of the content. … Very rich, detailed, and comprehensive metadata about that content will result without the large number of humans currently associated with these tasks.

At that point, the user will be presented with the text associated with the content. Each word, with exact reference to its precise positioning within the data stream, will be indexed. Manipulation of text (e.g. cut, copy, paste), will, in effect, be the method of editing that content. Picture and sound will follow along. [Emphasis mine]

Readers of my blog know that I think machine learning is going to revolutionize the edit suite; mainly by reducing the need for Assistant Editors to perform ‘mechanical’ tasks like Ingesting, Sync-ing, and Grouping. But I don’t agree with Ohanian here. And I think his point of view, that editing is basically mechanical, represents one of the problems we face when trying to discuss the future of nonlinear editing.

Editing is a visceral experience. Full stop.

Editing will never be as easy as cutting and pasting text because what’s being said is often secondary to how something’s said. Think about the Brett Kavanaugh hearings. You could read transcripts all day long, but his anger is what left its lasting impact.

The primacy of subtext is applicable to all genres of editing, from the biggest tentpole blockbuster to most corporate HR training video. Anyone who’s listened to multiple reads of Voice Over will know firsthand that the same words spoken differently feel very different each and every time. What makes every editor unique is how these subtle differences inform their creative process.

The source/record metaphor is probably a dated way to interact with audio/video media; and smarter tools that assist the editor in finding and selecting media are needed. But I think “Marking IN and Marking OUT to create edit points” is going to be with us for a while because Marking IN and Marking OUT is editing. The problem isn’t the model, it’s that we need to expand our definition of literacy to include video.

Recorder. A perfect Machine Learning use case.

Atlas Obscura and the New Yorker report on a new documentary about a remarkable woman, Marion Stokes, who recorded 70,000 (!!) hours of television on VHS tapes from 1975 until 2012.

Marion Stokes was secretly recording television twenty-four hours a day for thirty years. It started in 1979 with the Iranian Hostage Crisis at the dawn of the twenty-four hour news cycle. It ended on December 14, 2012 while the Sandy Hook massacre played on television as Marion passed away. In between, Marion recorded on 70,000 VHS tapes, capturing revolutions, lies, wars, triumphs, catastrophes, bloopers, talk shows, and commercials that tell us who we were, and show how television shaped the world of today. 

From the documentary’s website “RECORDER: The Marion Stokes Project”.

The 70,000 VHS tapes are currently awaiting digitization by the Internet Archive to be made available to the public. But these tapes also represent the ideal use case for Machine Learning technology like Google Vision to make it all searchable.

This also clearly demonstrates the need for a new editing metaphor, something like Tom Ohanian wrote about on his excellent State of Digital Nonlinear Editing series on LinkedIn.

Because a massive amount of people can read. And if they interact with content not first and foremost via video and audio, but with words, manipulation of content becomes really easy and very accessible. And it will / should work along these lines: Content that is recorded will then be processed by a variety of AI application suites. Each suite will provide different functionality (e.g. tonal analysis, speech-to-text, etc.) based on the characteristics of the content. When a live or recorded stream of content is digitized, it will be subjected to a variety of these suites.


At that point, the user will be presented with the text associated with the content. Each word, with exact reference to its precise positioning within the data stream, will be indexed. Manipulation of text (e.g. cut, copy, paste), will, in effect, be the method of editing that content. Picture and sound will follow along.

Tom Ohanian’s State of Digital Nonlinear Editing and Digital Media 10

(Note: Linkedin’s poor formatting makes these articles more difficult to read than necessary, but stick with it, his series is very insightful and thought provoking.)

look at the data — but not too much

Great interview with AMC’s Josh Sapan at Recode about the benefits and limitations of using data in the creative process:

https://megaphone.link/VMP9318038387

This mirrors what I’ve been saying about using data to inform the creative process, not decide. Too bad we don’t have the tools we need yet.

Further reading:

On Creative Fulfillment

One of the most important days in challenging how I think about work was in graduate school when I visited an adjunct professor at his consulting office on K St. in Washington, DC. He asked me what I did for work, and I told him that I was working at night as an Assistant Editor. His immediate response was, “Wow, I imagine that the level of frustration in your field is very high, because the jobs that are interesting probably don’t pay the bills, and the jobs that pay the bills probably aren’t very interesting.”

At this moment in time I’m very fortunate to have a job that’s engaging, interesting; and pays the bills! But this hasn’t been the norm for large portions of my career. And I don’t think I’m alone. One of the most common complaints I hear when I work with people on their resumes is how dissatisfied they are on their current show.

I believe that some of these frustrations are just an innate part of the entertainment industry. But perhaps there are ways we can learn to think about our jobs to help restore a sense of balance and value in our work.

The Business

The website FilmToolKit recently posted an infographic: Simple Ways to be the Best PA on Set. Number three is crushing:

Despite how much I hate this advice with every cell in my body I know that it is absolutely true. Everyone I’ve ever spoken to has seen someone less intelligent or less hardworking get ahead of them because of luck, and who they know. Hollywood is the place that invented the term ‘failing up’ afterall.

The fact of the matter is that Hollywood utilizes very little data in its decision-making process. Sure we track box office receipts or nightly viewers, but we don’t actually know why one show succeeds while another fails. I believe that this lack of data on the production process itself creates an environment where irrational decision-making is the only decision-making available, therefore people defer to their ‘gut’ and are lead by perception. This is all another way of saying the old adage:

“In Hollywood, nobody knows anything.”

William Goldman

I think there is also an element of self selection at play. Research has started to suggest that people who need consistency; where it is clear how promotion occurs and when the next paycheck will arrive, tend to gravitate towards stable fields such as civil service. While people who work in the cultural industries have a different set criteria for measuring job satisfaction and may even thrive due to the volatility a life in the arts is certain to entail.

“The TV business is uglier than most things. It is normally perceived as some kind of cruel and shallow money trench, a long plastic hallway where thieves and pimps run free and good men die like dogs, for no good reason.”

Hunter S. Thompson

Repose

Having a good sense of humor is probably the best way to thrive in the business. But failing wit, a more meditative approach might have more to offer the rest of us. The School of Life has written an assortment of articles about work. I highly recommend starting with this one on: The Creative Itch. Learning to identify which aspects of my job bring me the most joy has helped guide me towards new opportunities. It’s also given me insight into what I should steer clear of too!

You can also check out the The Sorrows of Work. This extended essay on the subject of work reminds us that our struggles are not unique, but a plight shared by the entire working class. There are no perfect jobs, because we work within an imperfect system.