look at the data — but not too much

Great interview with AMC’s Josh Sapan at Recode about the benefits and limitations of using data in the creative process:

https://megaphone.link/VMP9318038387

This mirrors what I’ve been saying about using data to inform the creative process, not decide. Too bad we don’t have the tools we need yet.

Further reading:

Scene Waste Analysis (Data Science Experiment Proposal)

I’m looking for feedback on this idea I have for a data science experiment. I think it could be one of the first experiments run in unscripted television production. If you are a data scientist who would be interested in partnering up, feel free to reach out to me on twitter: @lowbudgetfun

Hypothesis

Unscripted television production generates a lot of waste in the form of unused scenes. Scenes that are recorded and edited, but ultimately end up on the cutting room floor. These unused scenes represent wasted resources (time & money) and should be minimized. However, the creative process is mysterious and often requires playful exploration (trial & error), which is fundamentally at odds with the goal of minimizing waste. While eliminating waste completely is impossible, perhaps it is possible to reduce production waste by bringing certain trends to the attention of the story team during the planning process of subsequent seasons.

I hypothesize that by categorizing and analyzing all of the scenes that make it into a show, and comparing them to the scenes that are left out, trends could be identified that would enable Production Companies to avoid shooting similarly wasteful scenes in the future. In addition, I think that analyzing scene story summaries with a tool like Google’s Cloud Natural Language Sentiment Analysis could provide additional insight into why some scenes work, while others do it.

What is a Beat Board?

If you’ve visited a writing room you might have noticed a corkboard with lots of index cards on it. Informally known as a Beat Board; it is a tool writers utilize to help them visualize the structure and flow of the story they are trying to tell. Each index card represents one scene or story beat. By rearranging the cards on the board writers are able to quickly experiment with alternative story opportunities.

I first came up with the idea for this experiment after reading Lean Startup and learning about this methodology’s relentless pursuit of reducing waste:

The critical first question for any lean transformation is: which activities create value and which are a form of waste?

It was around that time when I walked into the EP’s office and observed that the number of cards representing scenes that didn’t make it into the show almost out numbered the cards of scenes that made it in. After chatting with colleagues on other shows, I anecdotally confirmed that this is not out of the ordinary.

What are Story Summaries?

This idea really clicked into place for when I learned about the influence data science is having on the humanities and I realized that many unscripted productions generate their own large body of textual material.

After a Reality Television crew records a scene for the day, a Field Producer will summarize, in writing, what happened. Story summaries can range from being a relatively objective description of an event, to impassioned prose about the cast’s feelings.

Sample Scene Summary

After work, Jennifer Duveen arrives home to find Barry (her husband) in the kitchen looking at listings of recent art acquisitions sent over from their secretary. He asks her how her day was and updates her on the status of their search. Jennifer comes over to take a look and they scroll through the most recent options and find a few great qualities but no true contenders. Disappointed, they try to see the good in some of the listings but quickly devolve into making fun of how they could sell these paintings if they had too. Barry mocks one of the gallery’s biggest clients who will buy anything that makes her look sophisticated, while Jennifer recalls that another client will buy anything so long as it’s mostly green, because it reminds him of money. With such witty banter between them, it’s obvious why these two are a good pair. Now, they decide it’s time to explore other options. Barry wonders aloud whether he should travel to a show in Miami, but Jennifer silently recoils. The silence between them is a silent reminder of the pain Jennifer feels after Barry’s affair. Without saying another word Barry retreats into his office with his laptop, while Jennifer pours another glass of wine.

Since the gap between when a scene was shot and when it begins editing can be several months, Story Producers will review a the story summary before they start the post production process of creating a Cutdown to handoff to the editor. Therefore, the value of good story summaries is also an underappreciated practice on many unscripted productions.

Scene Categories

In order to maximize the utility of this experiment, I think it will necessary to develop a system of scene categorization. Broadly, scenes can be separated into internal (Int.’s) and external (Ext.’s) locations. Scenes can also be organized by the number of cast members in them (1 – 7) and any additional people who appear on screen (family members or show ‘friends’). Scenes can also be categorized by their technical or production aspects, such as: cameraperson, field producer(s), or camera type (ENG or Car Camera, etc). A full taxonomy will be expanded upon in a future post.

Developing a Framework

The goal of this experiment is to develop a tool for unscripted story teams to use during pre-production while planning a season’s scenes. My hypothesis is that certain elements cause scenes to become unusable, but these elements are currently unknown. By identifying similarities between used and unused scenes, story teams will be able to reduce waste by avoiding things that won’t work.

A few years ago I wrote about my experience working on a competition show and manually digging through the sequences and budgets to uncover the cost of Loading, Grouping, and Storing the show’s car camera footage for the entire season was approximately thirty thousand dollars. And yet only one car scene made it into the entire twelve episode season. I’m not saying that eliminating the car cameras was the right decision. But I believe that it should have been discussed. Perhaps eliminating the car cameras would have freed up money for an additional challenge. Or an additional camera operator to gain additional coverage of the events. Or perhaps the savings could have been spent on an additional editorial team. These are all options that have vast creative implications.

We are on the cusp of having these trends brought to our attention with minimum friction. As we become more familiar with these types of analysis, our resistance lessens and we become empowered to make smarter decisions. I’d like to build the tools that enable these conversations.

How Netflix sent the biggest media companies into a frenzy

Good article on CNBC comparing Netflix and its competitors. Here is a particularly good quote about its spending practices:

While networks make decisions on TV ratings, Netflix plays a different game. Its barometer for success is based on how much it spent on a show rather than hoping every show is a blowout hit, said Barry Enderwick, who worked in Netflix’s marketing department from 2001 to 2012 and who was director of global marketing and subscriber acquisition. Since Netflix is not beholden to advertisers, niche shows can be successful, as long as Netflix controls spending. [Emphasis mine]

I’ve said it before, and I’ll say it again and again and again and again: (I could go on, but I won’t) the television production model needs to change!

The way television is produced is extremely wasteful. The creative process is always going to require exploration and false starts, it’s not an assembly line; but the amount of waste satisfying the Highest Paid Opinion alone is staggering. At a minimum the industry needs tools that enable more granular cost tracking, as well as tools that enable the analysis of that data. Just saying…

What do the Avengers talk about. An interesting use case for Data science and creative writing

Interesting post on Medium about analyzing the word usage of the Avengers. A more in depth how-to can be found here.

Avengers_Word_Usage
So where are the Guardians?

What I like about this analysis is that you can draw interesting inferences from the data:

Thor’s keywords suggest that in the Avengers movies (not including the films Thor headlines, like Ragnarok and The Dark World), he’s more action-oriented than most other characters. With the exception of his relationship with Loki, he tends to focus on tangible artifacts that drive the plot forward. Like Loki’s scepter, the Tesseract, and the mind stone.

Check out how Vision and Scarlet Witch have some similar words- they’re talking about fear an awful lot. I’m hoping they stay synced in Infinity Wars. Interestingly, I did a sentiment analysis as well, and Vision had the most lines with a negative sentiment. It’s not because he’s a constant downer, but because he calls situations like he sees them and reflects sometimes on the futility of the human heroes he comes to love. He sees the extra, extra big picture, and I get the sense it disturbs him.

This analysis is very from the less useful one I critiqued last November. As analytic tools become more prevalent, it is important that our understanding of their uses grows with it. Glad to see work like this being done.

 

NLE Analytics: 3 User Scenarios

Longtime readers know that I believe that our NLE’s are missing an entire dimension of tools. They are woefully inadequate when it comes to helping us make informed business decisions. A new class of tools I call NLE Analytics are required to enable Producers, at the production management level, to make informed decisions. It is my hope that the three user scenarios below prompt Product Managers at Adobe (Premiere), Apple (FCPX), Avid (Media Composer), and Blackmagic Design (Resolve) to take this need seriously, and consider adding these tools into future versions of their software.

STARWARS-REEL-1-940-1010
Reel One from Stars Wars The Last Jedi. SO much inaccessible data!

Evaluating Music Libraries:

Rene is the Post Supervisor of an unscripted show that was just renewed for its fourth season. Last season her editors complained that the show’s music library was beginning to feel stale. It is Rene’s responsibility to negotiate with the music vendors, but she doesn’t know which are providing value and which are resting on their laurels. Is she paying one music vendor more than another, but using their library less?

The current process is to ask the editors what they think; which is really asking them what they feel; subject to all of the cognitive biases that lead us into bad decision making. The other option available to Rene is to load the previous season’s music cue sheets into Excel and go through numerous contortions to arrive at a general quantitative number like: “music company X provided 32 of 100 cues used in episode 303.” However, this doesn’t provide a full picture. Perhaps a company that has a smaller number of cues are actually being used for longer duration? Or perhaps one company provided a lot of cues early in the season, but dropped off later, demonstrating a lack of library depth.

And yet all of this information is so close at hand. The final sequences of each episode can provide all of this data and more. The right implementation could create a music usage dashboard providing Rene with a real time snapshot of what music her editors are currently using. If music company X is under represented, Rene would be empowered to reach out mid season and let her under performing music vendors know that their royalty checks are going to be a lot lighter if they don’t step up.

Specialty Footage:

Robert is the Production Supervisor of a second season show. His camera department is always asking for the additional resources (i.e. money) required to set up complicated car cameras, time lapses, drone, and b-roll sliding shots. The additional manpower and camera gear adds up to several thousands of dollars each week of shooting. Yet when Robert watched the last season on TV it feels like most of this specialty footage was never used.

The Post Supervisor can tell Robert how much storage space the specialty footage takes up, but this number is a poor indicator of the true cost of loading and organizing and maintaining this footage. Maybe the Post Supervisor can provide Robert with something like, “we used 2 minutes of car camera footage in our 44 minute episode 303,” but there is no easy way of providing Robert with an exact usage for the entire season. Or definitely no way to broke down it done by camera type.

And yet all of this information is so close at hand. The final sequences of each episode can provide all of this data and more. The right implementation could create a camera usage dashboard. In addition to specialty footage, the dashboard could provide up to the minute information about which cameraman’s footage is being used, perhaps indicating crew troubles.

Anomalous footage:

Outliers are things that stand out from the crowd. In statistics they indicate data that needs attention for a variety of reasons. In NLE Analytics, a shot could be considered an outlier if surrounded by a majority of footage from a different date/time. For example; if a scene is made up of footage from April 1st, but one shot is from May 15th, this should prompt the producer to ask, “why?” This anomalous shot could indicate a pickup shot due to camera issues. But perhaps it was due to creative discovery during the editing process? Or maybe it was a pickup interview because someone forgot to ask the right questions during the sit down?

Ep 12 days after
Wouldn’t it be great if your NLE could generate information this easily?

Whatever the reason, having a tool that easily identifies anomalous footage and brings it to the producer’s attention would be a valuable start to rethinking the entire production process from pre-production through delivery. I’ve explored how this could be applied to interviews in my post about: What the Bravo docu-soap can learn from Netflix part 1 & part 2.

Coda:

When he founded Digital Domain, James Cameron said that he wanted to create a place, “where technology would not just serve, but actually inform the creative process” (emphasis mine). The idea of NLE Analytics is to create a tool set that informs the creative process by pointing out inefficiencies and diverting money to places that will enhance the creative vision.

Today’s non-linear editing systems are huge repositories of data. But this enormously valuable information is completely inaccessible to us. With the right tools Producers can make informed business decisions and join the creative conversation. The entire production process could be reconsidered; a necessity in a world of shrinking budgets and increased deliverables.

I have two hopes. Firstly, that Product Managers at the big 4 developers take this idea seriously and consider implementing these tools into future versions of their software. Secondly, that Producers take these ideas seriously and let the big developer know that these are tools they want to use, and that future business will go to the ones who make it happen.